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Abstract 


Recent  studies  have  shown  frequent  failures  in  the  visual  detection  of  changes,  even  if  changes  are 
large  and  anticipated.  Many  man-made  systems  and  processes  rely  heavily  on  visual  displays  to 
convey  information,  so  failures  in  the  detection  of  changing  information  have  implications  for 
human-computer  interface  design.  Operators  monitoring  computer  displays  in  safety-critical 
environments  such  as  air  traffic  control  centres,  nuclear  plants  or  hospital  emergency  rooms,  need 
to  monitor  and  keep  track  of  a  large  volume  of  information.  The  state  of  the  processes  can  change 
rapidly,  and  need  to  be  kept  within  given  safety  or  operational  limits.  In  those  cases,  missing 
changing  information  could  have  an  immense  human  and  economic  cost. 

In  this  report,  we  review  research  documenting  several  types  of  perception-“blindness”  known  as 
change  blindness,  inattentional  blindness,  comparison  blindness  and  repetition  blindness. 
Understanding  the  perceptual  and  attentional  factors  that  produce  these  ‘blindness’  might  inform 
the  theory  and  practice  of  computer  interface  design  in  order  to  create  visualizations  that  will 
enhance  change  detection  in  monitoring  tasks. 

Based  on  an  Emergent  Themes  Analysis  approach,  we  identified  5  effects,  or  properties,  common 
across  the  different  forms  of  change  blindness,  and  hence  the  likely  effects  that  designers  need  to 
consider  when  designing  visual  interfaces  to  reduce  it.  These  five  effects  are  (i)  the  effect  of  rate 
of  change,  (ii)  the  effect  of  eccentricity,  (iii)  the  effect  of  proximity,  (iv)  the  effect  of  significance, 
and  (v)  the  effect  of  task  relevance. 

Several  highlighting  techniques  used  to  enhance  visual  search  and  stress  important  information  in 
visual  displays  such  as  changes  in  colour,  intensity,  blinking;  and  boxing  have  not  been  fully 
successful.  We  propose  using  a  new  technology  known  as  the  Multi-Layered  Display  (MED)  to 
investigate  perceptual  depth  as  a  cue  to  enhance  change  detection.  Until  now,  perceptual  depth  as 
a  cue  for  change  detection  tasks  has  not  yet  been  investigated.  We  believe  that  the  additional 
information  provided  by  MED  will  improve  the  management  of  attentional  resonrces,  and  thus  lead 
to  an  improvement  in  change  detection.  Further  research  will  comprise  the  completion  of  three 
experiments  investigating  the  use  of  perceptual  depth  in  the  detection  of  changes  in  monitoring 
tasks  taking  into  account  the  inflnence  that  the  5  effects  have  on  change  detection. 
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Preamble 


Studies  during  the  past  decades  have  shown  that  we  are  unahle  to  detect  changes  under  certain 
circumstances.  Surprisingly,  we  are  blind  to  certain  changes  that  happen  in  our  field  of  view. 
There  are  several  types  of  perception-”hlindness”  known  as  Change  Blindness,  Inattentional 
Blindness,  Comparison  Blindness  and  Repetition  Blindness.  These  types  of  “blindness”  present 
significant  problems  especially  in  dynamic  operational  environments  where  operators  monitoring 
computer  displays  have  to  monitor  and  keep  track  of  a  large  volume  of  information  that  change 
rapidly,  and  which  need  to  be  kept  within  given  safety  or  operational  limits. 

Researchers  have  stated  several  plausible  causes  for  Change  Blindness  but  the  phenomenon  have 
not  yet  been  fully  explained.  Many  have  focused  their  studies  on  the  brain’s  internal 
representations  of  what  we  see,  while  others  have  attempted  to  study  our  inability  to  detect  changes 
by  analyzing  memory  and  attention. 

Based  on  an  Emergent  Themes  Analysis  approach,  we  have  identified  5  effects  or  properties  of 
Change  Blindness  which  should  be  taken  into  account  when  designing  interfaces  for  complex 
systems. 

This  report  is  structured  in  six  main  chapters: 

The  first  chapter  explains  the  different  types  of  perception-blindness.  It  attempts  to  give  a 
comprehensive  review  of  the  empirical  evidence  of  these  phenomena.  It  also  presents  other 
empirical  studies  done  in  operational  environments  in  which  change  blindness  occurs. 

The  second  part  looks  at  various  hypotheses  that  attempt  to  explain  our  inability  to  detect  changes. 
Several  of  them  deal  with  our  visual  representations,  while  others  claimed  that  attention  is  required 
to  detect  changes.  A  short  comparison  is  made  in  order  to  understand  the  factors  that  might 
enhance  change  detection  according  to  the  studies  that  have  been  conducted. 

The  third  section  covers  the  introduction  of  a  new  device  known  as  the  MED  or  Multi-Layered 
Display.  It  has  the  ability  to  present  increasing  amounts  of  data  due  to  the  way  it  is  built.  This 


6 


/DC  Technical  Report  2007-3-001 
BOARD  Award  Grant  No.  FA8655 -06- 1-3081 


section  presents  several  highlighting  techniques  that  have  been  used  in  the  design  of  computer 
interfaces  but  that  have  not  been  100%  successful  for  change  detection  tasks.  Perceptual  depth  as  a 
cue  to  enhance  change  detection  is  introduced  as  a  new  possible  way  to  enhance  change  detection. 
Previous  research  conducted  in  the  MLD  is  presented. 

The  fourth  part  presents  a  brief  overview  of  the  processes  of  perception  and  attention  that  come 
into  play  to  enable  us  to  perceive  in  depth.  It  also  presents  an  overview  of  several  depth  cues 
needed  for  depth  perception. 

The  fifth  section  covers  the  results  obtained  from  an  Emergent  Theme  Analysis.  From  this 
analysis  we  identified  5  effects  of  change  blindness  that  influence  the  design  of  computer 
interfaces.  The  5  effects  are:  (i)  the  effect  of  rate  of  change,  (ii)  the  effect  of  eccentricity,  (iii)  the 
effect  of  proximity,  (iv)  the  effect  of  significance,  and  (v)  the  effect  of  task  relevance. 

The  final  chapter  put  together  the  lessons  derived  from  previous  studies  and  our  analysis.  We 
attempt  to  provide  some  guidelines  for  computer  interface  designers,  especially  for  those  involved 
in  complex  domains. 
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Chapter  1 

Are  we  really  blind  to  changes? 

Contrary  to  our  belief  we  see  far  less  than  what  we  think.  Our  eyes  acquire  visual  information  in 
between  blinks  and  eye  saccades  and  most  of  this  information  is  very  volatile  since  we  just  need  it 
for  very  short  periods  of  time.  Studies  in  change  detection  and  visual  perception  have  shown  that 
we  are  “blind”  to  some  events  that  happen  straight  in  front  of  us  while  we  are  looking  at  them. 
These  failures  of  perception  have  raised  studies  about  different  types  of  induced-“blindness”  which 
are  known  as  Change,  Inattentional,  Comparison  and  Repetition  Blindness 

This  Blindness  become  extremely  problematic  if  operators  in  dynamic  complex  work  domains 
such  as  nuclear  plants,  air  traffic  control,  and  ambulance  dispatch  centres,  fail  to  notice  a  change 
that  occurs  in  the  display:  Missing  a  red  light  can  costs  lives.  Operational  environments  maintain 
large  volumes  of  rapidly  changing  information  which  need  to  be  kept  within  given  safety  or 
operational  limits.  These  phenomena  show  that  possibly  dangerous  events  occurring  in  full  view 
may  go  unnoticed  if  they  coincide  with  small  apparently  innocuous  disturbances  or  interruptions 
such  as  a  phone  call,  becoming  crucial  due  to  the  enormous  human  and  environmental  costs  that  a 
failure  in  such  systems  could  cause. 

Change  Blindness 

Change  Blindness  is  a  phenomenon  in  visual  perception  where  large  changes  within  a 
visual  scene  are  undetected  by  the  viewer  when  occur  during  a  visual  disruption  such  as  a 
flicker,  an  eye  movement,  a  saccade  or  a  movie-cut  (O'Regan  2000;  Simons  2000;  Rensink 
2002). 

This  phenomenon  was  first  explored  systematically  by  George  McConkie  and  his 
colleagues  in  the  late  1970’s  focusing  only  on  changes  introduced  to  words  and  text  during 
eye  movements.  Later,  John  Grimes  (1996)  studied  this  phenomenon  in  the  domain  of 
scene  perception  demonstrating  that  people  miss  large  changes  to  scenes  when  the  changes 
are  introduced  during  an  eye  movement. 
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O’ Regan  (2000)  showed  that  elements  of  the  picture  that  occupied  as  much  as  a  fifth  of  the 
picture  area  would  not  be  seen.  These  experiments  required  observers  to  view  high- 
resolution,  full-colour  daily  visual  scenes  presented  on  a  computer  monitor,  while  their  eye 
movements  were  being  measured.  The  application  was  programmed  to  make  changes  in 
the  scene  as  a  function  of  where  the  observer  looked.  For  instance,  when  the  observer 
looked  from  the  door  of  a  house  to  the  window;  the  window  (or  some  other  element  of  the 
scene  such  as  the  sky,  or  the  car  parked  in  front  of  the  house)  changed  either  disappearing, 
being  replaced  by  a  different  element,  changing  colour  or  position  (e.g.  Henderson  & 
Hollingworth,  1999;  McConkie  &  Currie,  1996  cf.  O'Regan  2000). 

It  was  also  demonstrated  that  when  observers’  eyes  were  directly  fixated  on  the  location  of 
change,  they  still  failed  to  detect  the  change  40%  of  the  time.  Thus,  direct  fixation  aids 
detection  but  does  not  ensure  it  Fixation  was  not  a  guarantee  of  detection  (Rensink, 
O'Regan  et  al.  1997). 

Experiments  showed  that  change  blindness  was  not  specifically  related  to  eye  movements. 
Rensink  et  al  (1997)  popularized  the  "flicker"  technique  in  which  two  images  alternate 
repeatedly  with  a  brief  (80ms)  blank  screen  after  each  image  (Figure  1).  It  was  found  that 
large  changes  could  be  made  to  the  scene  without  the  observer  noticing  when  the  blank 
screen  was  inserted.  On  the  contrary,  where  no  flicker  was  inserted  in  between  the 
pictures,  the  change  was  immediately  visible  and  totally  obvious.  (See 
http://www.usd.edu/psvc301/Rensink.htm  for  a  demonstration  of  these  videos) 
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Figure  1 :  Flicker  Paradigm 

Source:  http://www.cs.ubc.ca/~rensink/flicker/index.html 


A  variant  from  the  flicker  paradigm  is  the  mud-splash  technique:  Experimenters  flashed 
some  dot  patterns  located  sparsely  on  top  of  an  image  appearing  at  the  same  time  as  the 
change  occurs  but  without  covering  the  change  location  (Figure  2).  Although  responses 
were  considerably  faster  than  when  the  blank  screen  was  used,  people  failed  to  detect 
changes  (Rensink,  O'Regan  et  al.  1997;  Simons  and  Levin  1997b).  This  occurrence 
showed  that  possibly  dangerous  events  occurring  in  full  view  might  go  unnoticed  if  they 
coincide  with  even  small  apparently  innocuous  disturbances.  (See 

http://nivea.psvcho.univ-paris5.fr/ASSChtml/dottedline.gif  for  a  demonstration  of  these 
videos). 
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Figure  2:  Mud-splash  Technique 

Source:  http://www.cs.hris.ac.uk/~cater/PhD/ChangeBlindInfo/Examples.html 


Other  studies  showed  that  change  blindness  occur  when  the  change  is  introduced  during  a 
cut  in  a  motion  picture,  even  when  the  change  was  to  the  central  actor  in  a  scene,  67%  of 
observers  failed  to  detect  the  change  from  one  actor  to  another.  (Simons  and  Levin  1997a; 
Simons  and  Levin  1997b).  (See  http://viscog.beckman.uiuc.edu/grafs/demos/23.html  for  a 
demonstration  of  these  videos). 

In  a  compelling  real-world  demonstration  an  experimenter  asked  pedestrians  for  directions. 
While  the  pedestrian  was  providing  directions,  two  additional  experimenters,  carrying  a 
door,  passed  between  the  initial  experimenter  and  the  pedestrian.  During  this  brief 
interruption,  a  different  person  replaced  the  original  experimenter.  Even  though  the  two 
experimenters  looked  quite  different  and  had  distinctly  different  voices,  half  the  subjects 
failed  to  notice  that  they  were  talking  to  a  different  person  (1998)  (Ligure  3).  However, 
those  who  noticed  tended  to  be  from  the  same  social  group  (students)  as  the  experimenters, 
and  those  who  failed  to  notice  tended  to  be  older  than  the  experimenters.  (See 
http://viscog.beckman.uiuc.edu/grafs/demos/12.html  for  a  demonstration  of  the 
experiment). 

To  explore  this  in-group/out-group  effect,  they  conducted  a  second  experiment  in  which 
the  same  two  experimenters  were  dressed  as  construction  workers.  By  making  the 


/DC  Technical  Report  2007-3-001 
BOARD  Award  Grant  No.  FA8655-06-1-3081 


experimenters  members  of  an  out-group  for  the  younger  subjects,  change  detection  was 
reduced  to  only  35%.  However  when  both  the  direction-asker  and  the  direction-giver  were 
students,  change  detection  was  almost  100%. 


Figure  3:  Failure  to  Detect  Changes  to  People  in  Real-World  Interactions 
Source:  Varakin,  Levin  et  al  (2004) 

(Simons  and  Levin  1998).  Box  A:  initial  experimenter  approaches  a  pedestrian.  B:  the  initial  experimenter 
walks  away  behind  a  passing  door,  and  another  experimenter  finishes  the  interaction  (box  C).  D:  the 
switching  experimenters  side  by  side. 

Inattentional  Blindness 

Inattentional  Blindness  occurs  when  attention  is  diverted  to  a  specific  task,  observers  often 
fail  to  perceive  an  unexpected  object,  even  if  it  appears  at  fixation  (Mack  and  Rock  1999). 
Mack  and  Rock  (1998)  conducted  an  experiment  where  a  small  cross  was  shown  briefly  on 
a  computer  screen  for  each  of  several  experimental  trials  and  asked  participants  to  judge 
which  arm  of  the  cross  was  longer.  After  several  trials,  an  unexpected  object,  such  as  a 
brightly  coloured  rectangle,  appeared  on  the  screen  along  with  the  cross  (Figure  4).  They 
reported  that  participants — busy  paying  attention  to  the  cross — often  failed  to  notice  the 
unexpected  object,  even  when  it  had  appeared  in  the  centre  of  their  field  of  view.  When 
the  cross  did  not  divert  participants’  attention,  they  easily  noticed  those  unexpected  stimuli 
such  as  the  rectangle. 
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Figure  4:  Screen  shots  Inattentional  Blindness  Experiment 
Source:  Mack  and  Rock  1999  http://psvche.cs.monash.edu.au/v5/psvche-5-03-mack.html 


They  also  found  that  while  the  inattentional  blindness  effect  was  robust  for  meaningless 
stimuli,  their  participants  almost  invariably  noticed  when  their  own  names  were  presented. 
Even  more  amazingly,  inattentional  blindness  returned  if  the  participant’s  name  was 
slightly  misspelt. 


Arising  from  this  set  of  experiments,  they  supported  a  flexible  selection  theory  of  attention 
which  states  that  depending  on  the  nature  of  the  stimuli  (high-  or  low-level  attributes),  it 
may  be  that  the  system  operates  to  minimize  effort  and  so  will  select  on  the  basis  of  a  low- 
level  attribute  like  size  if  possible  but,  if  not,  will  process  the  input  more  deeply,  as  seems 
to  be  the  case  with  lexical  stimuli  (Mack  and  Rock  1999). 


If  the  critical  stimulus  falls  within  the  zone  of  attention,  the  probability  that  it  will  receive 
some  benefit  from  attentional  processing  seems  high.  If,  however,  the  critical  stimulus  has 
no  particular  intrinsic  signal  value  and  is  irrelevant  to  the  subject's  assigned  task,  the 
attentional  warrant  that  permits  a  stimulus  to  pass  from  implicit  to  explicit  perception  is 
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minimal,  so  that  only  its  presence  or  its  bare  bone  features  are  perceived.  If  the  same 
stimulus  were  to  fall  outside  the  zone  of  attention,  then  it  is  far  more  likely  that  even  its 
presence  will  go  undetected,  because  it  does  not  have  the  benefit  of  even  minimal 
attentional  processing. 

In  another  set  of  experiments,  participants  were  asked  to  view  dynamic  scenes  of  four 
white  and  four  black  shapes  (T’s  and  L’s  or  circles  and  squares)  independently  moving  on 
a  computer  screen  (Figure  5).  Periodically,  these  shapes  would  bounce  off  the  edge  of  the 
display.  The  participants’  task  was  to  count  the  number  of  times  shapes  of  a  designated 
colour  (either  black  or  white)  touched  the  edge  of  the  display  ignoring  shapes  of  the  other 
colour.  Each  trial  lasted  for  15  seconds.  On  the  third  trial,  an  unexpected  object  appeared 
from  the  right  side  of  the  display  and  moved  across  it,  remaining  visible  for  5  seconds.  The 
most  striking  result  was  that  less  than  half  the  observers  (47%)  noticed  the  unexpected 
object  when  it  passed  right  through  the  centre  of  the  screen,  even  though  the  object  always 
stayed  on  what  was  presumably  the  focus  of  attention  and  was  clearly  visible  for  5  seconds. 
(Most,  Simons  et  al.  2000) 

Consistent  with  the  findings  of  Mack  and  Rock  (1998),  Most  et  al  (2000)  results 
demonstrated  a  role  for  distance  from  the  focus  of  attention  (i.e.,  the  region  around  the 
horizontal  line).  The  farther  the  unexpected  cross  was  from  the  horizontal  line  (Figure  5), 
the  fewer  the  observers  who  noticed  its  appearance  on  the  critical  trial,  with  detection 
dropping  to  only  21%  noticing  overall  in  the  Very  Far  condition.  (See 
http://viscog.beckman.uiuc.edU/grafs/demos/6.html  for  demo) 


14 


/DC  Technical  Report  2007-3-001 
BOARD  Award  Grant  No.  FA8655-06-1-3081 


Very  Far 
Far 

Near 

On  Line 

Near 

Far 

Very  Far 


Figure  5:  Most  et  al.  2000  Experiment 
Source:  http://psvche.cs.monash.edu.au/v6/psvche-6-14-most.html 

In  1975,  Neisser  and  Becklen  conducted  a  different  set  of  Inattentional  Blindness’ 
experiments.  Participants  were  asked  to  monitor  one  of  two  overlapping,  simultaneous 
events  in  a  computer  display:  two  people  playing  a  hand  game  or  a  team  passing  the  ball  to 
each  other.  If  they  monitored  the  hand  game,  they  pressed  a  button  with  each  attempted 
slap.  If  they  monitored  the  ball  game,  they  pressed  the  button  for  each  pass  (Neisser  1976). 

On  the  last  four  trials,  an  unexpected  event  would  occur,  for  instance,  the  two  hand-game 
players  stopped  and  shook  hands  or  one  of  the  ball-game  players  threw  the  ball  out  of  the 
game  and  the  players  continued  to  pretend  to  be  passing  the  ball.  In  total,  50%  of  the 
subjects  showed  no  indication  of  having  seen  any  of  the  unexpected  events,  and  even 
subjects  who  did  notice  could  not  accurately  report  the  details  of  them.  Researches  argued 
that  the  video  superimposition  gave  the  scenes  an  odd  appearance  making  it  difficult  to  see 
as  they  would  without  the  superimposition. 

To  demonstrate  that  the  superimposition  was  the  cause  of  the  inattentional  blindness 
problem,  Simons  and  Chabris  (1999)  set  up  an  experiment  with  two  styles  of  video:  a 
transparent  condition,  replicating  Neisser  and  Becklen’ s  superimposition,  and  a  standard 
video  which  they  called  “opaque  condition”.  The  opaque  condition  showed  all  seven 
actors  simultaneously  causing  some  occlusions  between  actors  and  basketballs  (Figure  6). 
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The  video  lasted  for  75  seconds.  It  showed  2  teams  of  3  players  each  wearing  either  white 
or  black  T-shirts.  For  the  unexpected  event,  a  short  woman  dressed  up  as  a  black  gorilla  or 
a  woman  with  an  umbrella  was  recorded  walking  through  the  room.  The  unexpected  event 
lasted  for  5  seconds. 

Observers  were  asked  to  monitor  one  of  the  two  teams.  They  should  keep  either  a  silent 
mental  count  of  the  total  number  of  passes  made  by  the  attended  team  (the  Easy  condition) 
or  separate  silent  mental  counts  of  the  number  of  bounce  passes  and  aerial  passes  made  by 
the  attended  team  (the  Hard  condition). 

The  results  revealed  a  substantial  level  of  inattentional  blindness  for  these  dynamic  events 
confirming  the  basic  resnlts  of  Neisser  and  colleagnes.  Out  of  all  192  observers  across  all 
conditions,  only  54%  noticed  the  unexpected  event.  More  observers  noticed  the  gorilla 
event  in  the  opaque  condition  (67%)  than  in  the  transparent  condition  (42%).  The  authors 
suggested  that  for  those  monitoring  the  black  team  was  easier  to  detect  the  black  gorilla 
due  to  the  similar  colours.  However,  even  in  the  opaque  condition,  33%  of  observers 
failed  to  report  the  event,  despite  its  visibility  and  the  repeated  questions  about  it. 


Figure  6:  Gorillas  in  the  Midst  (Simons  and  Chabris  1999) 
Sonrce:  Varakin,  Levin  et  al.  2004 
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Comparison  Blindness 

Another  type  of  blindness  is  the  difficulty  in  comparing  two  simultaneously  present  stimuli. 
This  inability  to  compare  2  concurrent  stimuli  seems  to  be  due  to  transaccadic  memory 
limitations  in  tasks  requiring  saccades. 

Scott-Brown  et  al.  (2000)  stated  that  people  fail  to  notice  changes  in  tasks  where  two 
stimuli  are  presented  in  the  central  visual  field  within  a  single  fixation  and  where  there  are 
no  transients  present.  Studies  in  comparison  blindness  have  demonstrated  that 
comparisons  over  space  within  a  single  fixation  to  be  as  difficult  as  comparison  over  time 
showing  that  memory  is  not  essential  to  produce  change  blindness  (Wright,  Green  et  al. 
2000;  Scott-Brown,  Baker  et  al  2000) 
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Repetition  Blindness 

A  wide  variety  of  visual  phenomena  reported  in  the  literature  have  considered  an  effect 
known  as  repetition  blindness  which  refers  to  the  impairment  in  detecting  or  reporting 
repetition  of  visually  presented  stimuli  such  as  words,  alphanumeric  characters  or  pictures. 
Repetition  Blindness  has  typically  been  studied  using  rapid  serial  visual  presentation 
(RSVP)  in  which  a  sequence  of  items  is  presented  within  the  same  spatial  locus  at  a  high 
spatial  presentation  rate  (Kanwisher,  Kim  et  al.  1996;  Chun  and  Cavanagh  1997;  Shapiro, 
Driver  et  al.  1997).  If  the  character  they  are  looking  for  appears  twice,  participants  usually 
detect  only  one  of  them  and  fail  to  notice  the  other  one. 

In  conclusion,  recent  evidence  from  work  on  change,  inattentional,  comparison  and  repetition 
blindness  shows  that  under  precisely  controlled  timing  and  response  conditions,  observers 
sometimes  fail  to  detect  stimuli  that  are  otherwise  clearly  visible. 

Repetition  Blindness  contradicts  our  common  belief  that  repeated  items  are  easily  recalled.  It  is 
not  an  attentional  problem,  it  seems  to  be  a  retrieval  failure,  not  an  encoding  failure,  which  is  not 
relevant  in  the  study  of  change  detection  and  therefore  is  not  going  to  be  taken  into  account. 

Comparison  Blindness  analyzes  how  poor  we  are  at  comparisons  that  do  not  require  memory. 
Rensink  (2002)  makes  a  clear  distinction  between  change  and  difference  claiming  that  “change 
refers  to  the  transformation  over  time  of  a  single  structure  [while]  dijference  refers  to  a  lack  of 
similarity  in  the  properties  of  two  structures”  (p.  250).  Consequently,  since  we  are  interested  in 
change  and  not  differences,  we  will  not  explore  this  topic  any  further. 

Change  Blindness  and  Inattentional  Blindness  share  a  condition  of  inattention  that  prevents 
observers  to  become  aware  of  salient  visual  changes  in  their  visual  field.  Researchers  have 
suggested  that  these  experiments  showed  that  without  attention,  for  Inattentional  Blindness 
experiments,  we  often  do  not  see  unanticipated  events,  and  even  with  attention,  for  Change 
Blindness  experiments,  we  cannot  encode  and  retain  all  the  details  of  what  we  see. 
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Change  Blindness  in  Operational  Environments 

Studies  in  change  detection  have  been  performed  in  driving,  aviation  (Wickens  and  Muthard  2003), 
and  combat  environments  (DiVita,  Nugent  et  al.  2004).  They  generally  found  that  the  main 
predictor  of  change  detection  was  the  relevance  of  the  change  to  the  task  being  performed. 
Accuracy  improved  as  the  relevance  increased.  Lesser  effects,  such  as  the  number  of  objects 
monitored,  the  positioning  of  monitored  objects  and  the  method  of  presentation  were  also  found. 

Niklolic  and  Sarter  (2001)  investigated  pilot  performance  in  detecting  mode  changes  in  their  flight 
management  system  when  monitoring  peripheral  displays.  They  found  that,  using  the  peripheral 
display  that  performed  the  best,  pilots  still  missed  over  10%  of  the  changes,  and  that  this  rate 
substantially  increased  to  almost  20%  with  competition  for  visual  attention.  DiVita,  Nugent  et  al. 
(2004)  verified  the  magnitude  of  change  blindness  in  a  realistic  scenarios  that  simulated  tasks 
performed  by  naval  CIC  operators  showing  that  in  applied  work  settings,  diversion  of  attention 
creates  an  opportunity  for  changes  to  occur  to  unattended  computer  monitors.  For  instance,  for 
changes  in  airplane  course,  only  68%  of  changes  were  correctly  identified  on  the  first  selection. 

Podczerwinski  et  al  (2002)  demonstrated  that  detection  of  relevant  changes,  changes  that  cause  a 
potential  conflict,  in  both  traffic  and  weather  systems  was  superior  to  that  of  irrelevant  changes. 
Change  detection  was  only  at  50%  accuracy,  but  improved  by  changes  that  were  more  salient 
(spatially,  not  digitally  represented),  and  were  more  meaningful  (causing  a  conflict  with  the  flight 
path).  Changes  that  were  relevant  to  the  pilot’s  flight  planning  task  were  detected  nearly  four 
seconds  faster  than  irrelevant  changes. 

Wickens  and  Muthard’ s  (2003b)  suggested  that  change  detection  was  dominantly  driven  by 
meaningfulness.  They  used  dynamic  integrated  hazard  displays  which  are  able  to  present  pilots 
with  the  changing  status  of  traffic  and  weather  overlaid  upon  the  dynamic  information  related  to 
route  guidance.  They  also  assessed  the  influence  of  the  monitor  size,  highlighting  levels,  and 
event  eccentricity  in  surveillance  tasks. 

Wickens  and  Muthard  (2003)  hypothesised  that  physical  differences  in  a  cockpit  hazard  display 
influence  the  ability  to  detect  changes  and  this  influence  itself  is  modulated  by  top-down  influences 
of  change  relevance  or  importance.  In  their  experiments,  pilots  were  asked  to  detect  changes  in  the 
movement  or  altitude  of  weather  systems  or  traffic  aircraft,  which  were  represented  in  integrated 
hazard  displays. 
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They  examined  change  detection  as  a  function  of  the  distance  of  the  event  from  own- ship,  known 
as  eccentricity  or  distance  from  the  foveal  vision,  and  the  presence  of  the  event  in  a  highlighted  or 
low-lighted  hazard  domain.  Analyses  revealed  that  change  detection  was  superior  for  events  that 
were  in  the  highlighted  display  database  and  that  performance  was  slightly  degraded  for  more 
eccentric  events.  In  the  highlighted  domain  of  the  display,  either  traffic  or  weather  information, 
search  was  found  to  focus  more  frequently  on  the  highlighted  elements,  and  to  a  lesser  extent,  to 
the  elements  that  were  located  more  closely  to  the  centre  of  focused  attention,  both  of  which  are 
consistent  with  models  of  visual  search  and  strategies  of  map  search  (  cf.  Wickens,  Muthard  et  al. 
2003).  Therefore,  since  highlighting  and  location  correlate  with  meaningfulness  in  the  aviation 
domain,  it  is  more  likely  that  a  changed  element  in  a  low-lighted  display  domain  or  in  the 
periphery  will  go  unnoticed  or  be  noticed  at  greater  latencies. 

When  assessing  the  role  of  computer  screen  size,  results  showed  that  change  detection  was 
unaffected  by  the  size  of  the  screen,  however,  although  not  statistically  significant,  accuracy 
revealed  a  meaningful  degradation  in  performance  for  small  displays  likely  because  of  the 
minimisation  of  the  display  elements  which  decreased  legibility  and  resolution. 

For  elements  located  in  the  periphery,  performance  was  again  degraded  for  changes  located  near 
the  perimeter  of  the  display,  probably  because  in  the  aviation  domain,  event  eccentricity  is 
confounded  with  relevance:  In  the  case  of  a  CDTI  (Cockpit  Displayed  Traffic  Information),  events 
(changes)  closer  to  own-ship  generally  are  more  relevant  to  flight  safety  (Wickens,  Muthard  et  al. 
2003b).  These  findings  imply  that  surveillance  of  the  display  perimeters  will  depreciate  and 
additional  methods  should  be  used  to  ensure  that  attention  is  sufficiently  directed  to  these  areas. 

Finally,  attention  guidance  aids  have  shown  to  assist  performance  in  directing  attention  to  the 
relevant  elements  of  a  scene,  which  according  to  the  authors  is  beneficial  in  improving  plan 
selection  accuracy  and  confidence,  especially  in  the  high  workload  conditions  by  improving 
situation  awareness  (Wickens  and  Muthard  2003).  Negative  effects,  such  as  cognitive  tunnelling, 
are  also  associated  with  imperfect  automation.  Consequently,  if  an  aid  fails  to  highlight  a  relevant 
element  of  the  visual  scene,  the  pilot  may  fail  to  detect  the  important,  but  uncued  element. 
Imperfect  automation  guidance  resulted  in  greater  frequency  of  missed  critical  events  (61%  misses 
with  imperfect  automation  vs.  39%  misses  in  control  condition).  Analyses  of  ASRS  (aviation 
safety  reporting  system)  reports  have  provided  evidence  of  monitoring  failures  linked  to  excessive 
trust  in,  or  over-reliance  on  automated  systems  (Wickens  1998) 
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These  studies  demonstrated  that  change  blindness  is  likely  to  occur  in  operational  environments. 
We  have  to  keep  in  mind  that  even  when  the  computer  screen  size  does  not  affect  change  detection 
performance,  eccentricity  and  intensity  of  the  elements  in  a  visual  display  have  a  significant 
influence  on  change  detection.  On  the  other  hand,  if  automation  aids  are  implemented  in  change 
detection  tasks,  they  have  to  be  very  precise  to  prevent  bias  and  untrustworthiness. 
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Chapter  2 

Why  does  change  blindness  occur? 


The  answer  to  this  question  is  not  straightforward.  The  current  hypotheses  can  not  explain  by 
themselves  all  the  change  blindness  effects  that  have  been  identified. 

For  centuries,  scholars  assumed  the  need  for  precise,  veridical  representations  of  our  visual  world. 
However,  change  blindness  studies  have  provided  evidence  against  the  existence  of  detailed 
internal  models  (Simons  and  Levin  1997b;  Noe,  Pessoa  et  al.  2000),  instead  change  blindness 
supports  the  phenomenal  experience  of  continuity  by  not  preserving  too  much  information  from 
one  view  to  the  next  (Simons  and  Levin  1997,  p.  267). 

In  Simons’  review  of  the  current  approaches  of  Change  Blindness  (2000),  he  presents  5  plausible 
causes  based  on  the  nature  of  our  visual  representations  (Figure  7)  stating  a  wide  range  of 
hypotheses  from  the  extreme  belief  that  none  of  our  visual  representations  are  stored,  to  the  belief 
that  all  of  our  visual  representations  are  stored  but  are  not  compared  until  something  triggers  the 
comparison.  These  five  hypotheses  are  detailed  below: 


Overwriting  Hypothesis 

The  most  plausible  explanation  for  change  blindness  is  that  the  initial  visual  representation  is 
simply  overwritten  or  replaced  by  the  blank  interval  or  by  the  subsequent  image.  Overwriting 
models  have  been  used  to  explain  visual  masking  as  well  as  poor  recognition  of  scenes  from  RSVP 
(rapid  serial  visual  presentation)  streams  (cf.  Simons  2000).  “Information  that  was  not  abstracted 
from  the  initial  scene  is  simply  replaced  in  the  representation  by  the  new  scene”  (p.  8) 


Gist/First  Impressions 

The  First  Impressions  hypothesis  supports  the  idea  that  observers  encode  the  features  of  the  initial 
object  or  scene  and  fail  to  encode  the  details  of  the  changed  scene.  Observers  “encode  the  gist  of 
the  scene  and  ignore  visual  details.  As  long  as  the  gist  remains  the  same,  change  detection  seems 
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unlikely  because  observers  have  not  expended  tbe  effort  to  encode  more  details”  (Simons  and 
Levin  1997b,  p.  266).  To  verify  that  tbe  meaning  is  constant,  observers  do  not  need  to  re-encode 
all  of  the  details  of  a  scene  with  each  view,  but  to  check  a  few  features  to  make  sure  they  are  seeing 
the  same  scene.  By  identifying  the  scene’ s  meaning,  they  can  avoid  the  need  to  encode  and  update 
all  the  details  related  to  that  schema  from  one  view  to  the  next.  If  a  change  affects  the  meaning  of 
a  scene,  observers  will  be  more  likely  to  compare  the  details  of  the  scene  to  their  representations 
and  thereby  detect  tbe  change.  The  meaning  serves  as  a  trigger  for  spontaneous  detection  of 
unexpected  changes.  This  hypothesis  is  supported  by  the  movie-cut  experiments  in  which  subjects 
who  failed  to  detect  a  change  in  the  central  object  in  motion  pictures,  described  the  features  of  the 
object  in  tbe  initial  scene  rather  than  in  the  changed  view  (Simons  and  Levin  1997a). 


Nothing  is  stored 

A  number  of  theorists  have  claimed  that  nothing  is  stored  and  the  world  serves  as  a  memory  store 
(see  Gibson  1979;  Simons  2000,  p.  10).  Given  that  none  of  the  details  of  the  first  image  are 
represented  in  a  visual  store,  change  detection  should  be  impossible  without  abstraction.  A 
somewhat  weaker  form  of  this  model  suggests  that  some  detail  is  preserved  between  scenes  (e.g. 
tbe  details  of  the  objects  in  the  viewer's  focus  of  attention).  In  this  way,  we  are  blind  to  change 
unless  it  affects  our  abstracted  knowledge  of  the  scene. 


Everything  is  stored 

Another  possibility  is  that  details  about  each  new  scene  are  stored,  but  cannot  be  accessed  until  an 
external  stimulus  forces  the  access.  The  visual  cognitive  system  may  assume  the  views  are 
consistent  unless  something  about  the  meaning  of  the  scene  triggers  comparison  (Simons  2000,  p. 
10).  For  instance,  in  one  of  the  adaptations  of  Simons  and  Levin’s  real-world  experiments,  the 
person  who  asked  for  directions  was  holding  a  basketball  that  was  taken  away  by  a  group  of 
students  who  passed  between  the  pedestrian  and  the  experimenter.  Only  three  of  the  subjects 
spontaneously  reported  the  disappearance  of  the  basketball.  When  the  remaining  participants  were 
asked  specifically  if  the  experimenter  used  to  have  a  basketball,  more  than  half  say  yes.  Subjects 
were  initially  blind  to  the  change,  but  when  prompted,  they  could  recall  the  presence  of  the 
basketball  and  its  features. 
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Combination  of  features 

A  final  hypothesis  is  that  details  from  an  initial  view  might  be  combined  with  new  features  from  a 
second  view  to  form  a  combined  “coherent”  representation  of  the  scene.  Presumably,  viewers 
would  not  be  aware  of  which  parts  of  their  mental  image  come  from  the  first  scene,  and  which 
come  from  the  second.  The  details  being  combined  must  make  sense,  and  must  be  consistent  with 
the  viewer's  abstract  understanding  of  the  scene,  otherwise  the  change  will  be  recognized  as 
"impossible"  or  "out  of  place". 


5  Causes  of  Change  Blindness 


Figure  7:  Current  Approaches  to  Change  Blindness 
Source:  Visual  Cognition  (Simons  2000) 


Simons  and  Levin’s  review  (1997)  presented  an  example  of  a  busy  city  street  in  which  several 
changes  occur  during  the  normal  set  of  events:  We  could  see  people  walking  in  a  busy  crowd, 
going  behind  cars,  kiosks,  telephone  booths;  someone  shifting  a  handbag  to  the  other  shoulder, 
someone  taking  a  mobile  phone  out  of  a  pocket  and  so  on. 

In  such  a  rapidly  changing  environment,  if  our  visual  system  would  encode  in  detail  every  change, 
it  would  become  very  confusing  and  chaotic.  Instead  our  visual  system  seems  to  focus  on  the 
information  we  need  to  know  while  ignoring  uninteresting  object  property  information.  This 
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example  raises  the  question  if,  in  fact,  we  really  have  time  to  check  the  environment  we  perceive 
continuously  for  changes  that  have  no  meaning  and  do  not  change  important  affordances. 

On  the  other  hand,  Rensink  support  the  coherence  theory  which  states  that  focus  attention  is 
required  for  change  detection  and  it  acts  as  a  hand  that  ‘grasps’  several  volatile  structures  called 
proto-objects  that  describe  several  aspects  of  a  scene  structure  (Rensink  2000).  This  interaction 
between  attention  and  proto-objects,  allows  attended  proto-object  properties  to  be  held  in  a 
coherent  form,  both  in  time  and  in  space.  When  this  loop  is  broken,  “coherence  dissolves,  with  the 
previously  attended  proto-objects  reverting  to  a  volatile  state.  A  change  in  a  stimulus  can  be  seen 
only  if  it  is  given  focused  attention  at  the  time  the  change  occurs”  (Rensink  2000,  p.  20).  Since 
only  a  small  number  of  items  can  be  attended  at  any  time,  most  items  in  a  scene  will  not  have  a 
stable  representation  (Pashler  1998)  provoking  change  blindness  if  attention  was  not  automatically 
directed  to  the  change. 

Thus,  vision  enables  us  to  learn  about  the  environment  but  there  is  a  good  deal  that  goes  on  around 
us  that  we  fail  to  notice.  The  reasons  why  we  are  unable  to  detect  some  changes  are  still  open  to 
question.  Over  the  past  years,  researchers  have  focused  increasingly  on  change  blindness  as  a 
means  to  examine  the  nature  of  our  representations.  Failure  to  detect  changes  provided  evidence  of 
the  absence  of  richly  detailed  internal  representations.  The  debate  is  still  open  if  whether  the  world 
serves  as  a  memory  store  (Gibson  1979,  Simons  2000),  or  our  visual  system  encode  some  or  all  the 
features  but  fail  to  make  them  available  to  memory  or  to  verbal  report  (Noe,  Pessoa  et  al.  2000). 
Rensink  and  colleagues  (1997)  assume  that  attention  functions  as  a  bottleneck  on  what  is  encoded 
but  Noe  et  al.  (2000)  argued  that  perhaps  memory  or  other  access  limitations  provide  a  bottleneck 
on  what  information  can  be  used  in  making  reports  about  the  detection  changes. 
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Chapter  3 

Perceptual  Depth  as  an  alternative 
technique 


So  far  we  have  reported  the  change  blindness  phenomena,  its  empirical  evidence  and  the  possible 
causes  for  this  “blindness”  to  occur.  We  understand  that  in  operational  environments,  namely 
safety-critical  ones,  being  blind  to  changes  that  might  occur  in  a  computer  display  could  cause 
immense  economic  or  human  costs. 

We  want  to  take  a  different  step  by  proposing  the  use  of  a  new  physical  device  that  due  to  its 
properties  might  enhance  change  detection.  This  new  device  is  called  the  MLD  (Multi-Layered 
Display).  Current  3D  computer  graphics  systems  include  perspective,  stereoscopic,  rotating,  head- 
motion  tracking,  holographic,  and  multi-planar  displays.  All  these  three-dimensional  technologies 
present  high  costs,  the  necessity  of  additional  glasses  or  additional  input/output  devices  and 
intensive  computational  requirements,  which  does  not  make  them  a  viable  option  to  implement  in 
safety  critical  environments  or  other  surveillance  and  operation  rooms.  However,  we  propose  that 
transparent  depth  displays  such  as  the  MLD  form  a  promising  development  because  it  overcomes 
these  drawbacks.  Additionally,  it  provides  a  perceptual  depth  cue  that  can  be  used  to  enhance 
change  detection;  however,  this  hypothesis  requires  further  evaluation. 

Visual  displays  often  use  transients  (detectable  visual  cues  that  signal  a  change  in  the  environment 
over  time)  -  techniques  such  as  highlighting,  changing  colours,  boxing,  reverse  video  or  flashing 
have  been  used  to  notify  users  about  important  events.  However,  recent  research  and  operational 
experiences  in  complex  domains  suggest  that  those  design  approaches  are  not  always  successful. 


The  highlighting  paradox 

Many  different  techniques  have  been  used  to  make  objects  in  a  computer  display  more  salient  and 
more  noticeable.  Researchers  have  studied  techniques  such  as  boxing,  reverse  video,  blinking,  use 
of  different  colours  and  different  levels  of  intensities  (bold).  Studies  have  shown  contradictory 
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results.  While  Gomberg  (1985)  (cf.  Fisher  and  Tan  1989)  conducted  an  experiment  in  which 
subjects  were  required  to  search  for  a  single  target  digit  in  a  background  of  4  distracter  digits. 
Results  demonstrated  that  participants  were  slower  on  average  to  find  the  target  in  each  of  the 
highlighting  conditions:  boxing,  blinking  and  reverse  video  compared  to  a  standard  condition.  The 
negative  outcome  for  the  highlighting  conditions  might  have  occurred  because  it  takes  longer  on 
average  to  identify  the  blinking  digit  when  it  is  off;  and  the  boxing  and  the  reverse  video  might 
have  caused  a  lateral  masking  that  delayed  the  identification  process. 

Fisher  and  Tan  (1989)  replicated  Gomberg’ s  experiment  using  colour,  reverse  video  and  blinking 
as  the  highlighting  conditions.  They  demonstrated  that  colour  did  not  create  lateral  masking  or 
temporal  delays,  and  clearly  attract  the  attention  almost  immediately.  Flowever,  Fisher  and  Tan 
(1989)  reported  that  performance  depend  on  the  validity  which  is  the  proportion  of  trials  on  which 
the  highlighting  correctly  indicated  the  target.  When  the  validity  was  only  50%,  response  times  to 
detect  the  target  in  the  colour  condition  was  faster  than  reverse  video  or  blinking  but  did  not  help 
participants  find  the  target  faster  than  in  the  control  trials  (no  highlighting).  When  validity  was 
increased  to  100%,  the  colour  condition  presented  the  fastest  response  times  overall,  participants 
took  192  ms  less  in  the  colour  condition  than  in  the  control  one,  and  207  ms  faster  than  the  colour 
condition  when  the  highlighting  validity  was  50%. 

Since  it  was  not  clear  from  Fisher  and  Tan’s  (1989)  experiment  what  would  occur  at  other  levels  of 
validity,  Tamborello  and  Byrne  (2006)  replicated  the  experiment  and  extended  the  resnlts.  They 
only  examined  colour  (red)  as  the  highlighting  condition  but  at  9  different  validity  levels:  0%, 
12.5%,  25%,  37.5%,  50%,  67.5%,  75%,  82.5%  or  100%.  There  were  three  conditions:  control  (no 
highlighting),  valid  highlighting  (target  -red)  and  invalid  highlighting  (distracter-red).  Results 
showed  that  subjects  became  faster  on  valid  trials  and  slower  on  invalid  ones,  which  means  that 
sensitivity  increased  as  validity  increased.  For  instance,  subjects  in  the  87.5%  validity  percentage 
condition  were  201  ms  more  sensitive  to  trial  validity  than  were  subjects  in  the  12.5%  condition 
(Tamborello  and  Byrne  2006).  This  indicates  that  subjects  learned  rapidly  whether  or  not  they 
could  take  advantage  of  the  highlighting  in  their  visual  search. 

Another  technique  employed  to  highlight  important  information  has  been  the  use  of  different 
intensities  as  a  pre-attentive  cne  for  selective  filtering.  Some  studies  that  have  focused  on 
operational  environments  such  as  air  traffic  control  used  intensity  techniques  to  de-clutter  maps  by 
“low-lighting”  certain  information  and  keeping  the  rest  at  a  higher  intensity  level  (Wickens, 
Ambinder  et  al.  2004).  Consistent  with  visual  search  models,  when  more  information  is  presented 
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in  the  display,  the  time  to  detect  an  element  increases  as  the  amount  of  clutter  does.  One  way  to 
reduce  the  penalty  of  increased  clutter  in  a  map  search  is  to  increase  the  visual  salience  of 
particular  domains  of  information.  Wickens,  Ambinder  et  al.  (2004)  showed  that  increasing 
intensity,  increases  the  salience  and  attention-capturing  properties  of  the  items  in  a  map,  however 
the  intensity  difference  does  not  have  to  be  to  large  to  achieve  adequate  levels  of  discriminability 
since  participants  processed  any  dissimilarity  between  intensities  as  a  categorical  difference.  The 
experimenters  suggested  that  intensity  coding  techniques  support  both  focused  and  divided 
attention  tasks  in  a  map  display. 

Highlighting  techniques  such  as  boxing,  reverse  video,  blinking  or  use  of  different  intensities  have 
not  proven  to  be  100%  successful,  therefore  we  propose  the  use  of  a  perceptual  depth  cue  to  make 
information  more  noticeable.  This  perceptual  depth,  still  to  be  examined,  is  obtained  by  using  a 
new  display  technology  called  the  MLD. 


Multi-Layered  Display  (MLD™) 

The  MLD  is  a  new  display  technology  that  incorporates  2  LCD  panels,  one  in  front  of  the  other 
separated  by  a  Perspex  layer  giving  it  the  potential  to  present  physical  depth.  The  MLD  is  not  a 
three-dimensional  display,  however,  it  has  the  capability  of  using  perceptual  depth  to  present 
increasing  amount  of  information  on  a  monitor  of  a  given  display  size,  and  more  importantly,  it  has 
the  potential  of  increasing  the  informativeness  of  the  presented  data  in  subtle  ways. 

It  is  important  to  mention  that  the  MLD™  is  an  affordable  technology,  easy  to  acquire,  in  contrast 
with  other  stereo-display  technologies  that  raise  formidable  costs,  sophisticated  high-speed  graphic 
software  and  acceptance  barriers  in  operational  environments.  Due  to  its  layering  capability,  the 
MLD  is  able  to  de-clutter  information;  we  assume  that  it  may  reduce  the  number  of  orthographic 
displays  currently  required  in  surveillance  and  control  rooms.  Additionally,  the  MLD  makes  use  of 
the  WIMP  paradigm  (Windows,  Icons,  Menu  and  Pointer)  which  is  commonly  used  by  most 
computers  users  compare  to  other  3D  displays  that  may  require  replacing  the  mouse  and  keyboard- 
based  interactions  with  ad-hoc  3D  input  devices  (Tavanti  2004). 

However,  the  MLD  presents  some  disadvantages:  Its  novelty  produces  unfamiliarity  that  will 
require  additional  costs  on  training  and  time  to  get  use  to  it.  Designers  should  pay  special  attention 
to  how  elements  are  positioned  in  the  visual  display  due  to  a  “depth  ordering”  effect  that  cause 
objects  in  the  front  layer  to  be  occluded  by  other  objects  located  in  the  same  layer,  which  at  the 
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same  time  become  transparent  so  objects  in  the  back  layer  remain  visible.  Finally,  the  way  light  is 
reflected  through  both  layers  slightly  changes  the  hue  of  the  colours  presented  in  the  visual  display. 
Although  it  is  not  a  major  difference,  it  might  become  problematic  especially  when  using  a  dark 
background  on  the  back  layer. 

Although  the  MLD  has  some  disadvantages,  they  can  be  overcome  with  an  interface  design  that 
tackles  these  factors.  Its  capabilities  such  as  layering  and  de-cluttering  become  an  outstanding 
advantage  over  conventional  displays  which  strengthen  its  practicality  and  the  possibility  to 
introduce  it  in  surveillance  and  control  environments. 

It  has  been  shown  in  studies  on  the  MLD  that  it  can  improve  human  performance  in  visual 
information  search  (Duenser,  Billinghurst  et  al.  2006;  Duenser,  Mancero  et  al.  2006),  association 
and  comparison  tasks  especially  in  monitoring  behaviour  and  supervisory  control  (Joyekurun, 
Wong  et  al.  2005;  Wong,  Mansour  et  al.  2005).  These  studies  have  evaluated  factors  such  as 
legibility,  layering,  visual  search  and  multiple-object  tracking. 
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Studies  conducted  on  the  MLD 

Several  studies  have  been  conducted  during  the  past  years  since  Pure  Depth  created  the  MLD. 
Fred  Angelopoulos,  CEO  of  the  Redwood  Shores  (California)  start-up,  affirmed  that  “when  stuff 
moves  towards  us  we  naturally  react  quicker”(TechNow  2001).  This  claim  has  partially  been 
demonstrated  in  studies  conducted  in  the  MLD  when  comparing  participants’  performance  to  a 
single-layered  display  (SLD).  Although  results  have  been  contradictory,  they  have  provided 
important  insights  on  design  techniques  for  the  MLD. 

Visual  Search  Tasks 

A  number  of  studies  have  demonstrated  that  depth  can  be  used  as  a  segregation  cue  to  separate 
target  from  distracters  (cf.  Wickens  and  Hollands  2000;  Wong,  Mansour  et  al.  2005).  The  results 
showed  that  the  search  times  were  faster  when  the  target  and  the  distracters  were  separated  in 
different  depth  planes. 

An  experiment  was  conducted  to  investigate  whether  the  depth-information  provided  by  the  MLD 
would  lead  to  a  better  performance  in  a  visual  conjunction  search  task  compared  to  single-layered 
displays.  Twenty  participants  were  asked  to  detect  a  target  (red  circle)  among  several  distracters 
which  differ  in  colour,  shape  or  depth.  The  results  showed  that  depth  cues  alone  behaved  similarly 
to  search  times  with  shape  or  colour  cues  alone  in  which  regardless  the  number  of  stimuli,  the  time 
remained  almost  constant  but  participants  were  twice  as  slow  compared  to  the  colour  condition. 
However,  when  combining  depth  with  other  cues  such  as  colour  and  shape,  reaction  times  became 
faster  due  to  parallel  processing  (Duenser,  Billinghurst  et  al.  2006). 

Legibility 

A  study  investigated  legibility  at  different  levels  of  transparency  and  colour  combinations  for 
visual  perception  tasks  (Nees,  Villanueva  et  al.  2003).  Forty  students  participated  in  this 
experiment.  They  were  presented  with  a  random  series  of  six  different  readings,  each  with 
different  text  colour  and  background  pairings.  The  readings  were  either  presented  in  a  traditional 
single-layered  display  (control)  or  in  the  MLD  with  different  levels  of  transparency  (no 
transparency,  30%  and  70%).  Reading  speeds,  error  detection,  comprehension  speeds  and 
accuracy  were  measured.  The  results  showed  that  no  further  transparency  should  be  applied  to  the 
MLD  although  it  was  shown  that  a  higher  percentage  of  transparency  can  enhance  tasks  such  as 
error  detection.  It  was  also  demonstrated  that  foreground  and  background  colour  pairings  of  colour 
text  that  have  lead  to  poor  legibility  on  conventional  displays,  provoked  faster  reading  times  in  the 
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MLD,  emphasizing  the  importance  of  reassessing  design  recommendations  for  the  MLD  (Nees, 
Villanueva  et  al.  2003). 

Information  Layering  and  De-cluttering 

Participants  were  presented  with  several  moving  circles  labelled  with  a  specific  height.  They  had 
to  identify  all  the  moving  circles  in  a  certain  designated  height  for  the  easy  level,  or  identify  those 
moving  circles  that  were  above  a  certain  height  for  the  more  difficult  level.  The  moving  circles 
were  presented  either  in  a  single-layered  display  (SLD)  (control)  or  in  two  layers  using  the  MLD, 
and  in  different  transparency  levels  (0%,  30%,  60%  or  90%).  The  results  showed  that  for  the  easy 
level,  participants’  response  time  on  the  MLD  were  only  slightly  better  than  on  the  SLD.  However, 
for  the  more  demanding  task,  response  times  for  the  MLD  were  superior  than  their  counterparts  in 
the  SLD  at  all  levels  of  transparency  provoking  faster  response  times  ranging  between  1.5  to  4.5 
seconds  (Joyekurun,  Wong  et  al.  2005).  However,  because  there  were  only  six  students 
participating  in  this  study,  it  is  difficult  to  generalize  from  these  results. 

Another  study  focused  on  de-cluttering  displays  for  emergency  ambulance  dispatch  centres 
redesigned  the  current  map-based  dispatch  display  for  the  MLD.  The  information  locating  the  map 
and  other  contextual  information  was  shown  in  the  back  layer  and  only  significant  incident¬ 
relevant  information  on  the  front  layer  (Hayes  2006).  The  new  design  using  the  MLD  was 
evaluated  with  forty  students  where  they  were  presented  with  4  different  levels  of  difficulty. 
Participants  had  to  choose  a  centre  to  dispatch  an  ambulance  based  on  the  location  of  the  incident 
and  the  location  of  the  centre  where  they  were  sending  the  ambulance  from. 

For  the  first  level  of  difficulty,  participants  had  only  one  appropriate  centre  in  the  area  from  which 
to  dispatch  an  ambulance.  In  the  most  difficult  level  (level  4),  participants  had  to  balance  the 
ambulance  coverage  over  an  area  so  that  there  won’t  be  ‘gaps  in  coverage’  as  a  result  of  sending 
ambulances  to  an  incident  (Hayes  2006). 

The  results  of  this  study  did  not  present  any  significant  difference  between  the  MLD  and  the  SLD. 
However,  in  the  fourth  level  of  difficulty,  participants  made  more  errors  using  the  MLD  than  the 
SLD.  The  author  justified  this  higher  percentage  of  errors  by  a  phenomenon  called  “tunnelling  of 
vision”  since  the  participants  chose  the  incorrect  ambulance  dispatch  centre  because  they  did  not 
observe  a  river  illustrated  in  the  map  that  presented  an  obstruction  in  the  way  the  ambulance  had  to 
take  in  order  to  get  to  the  incident. 


31 


/DC  Technical  Report  2007-3-001 
BOARD  Award  Grant  No.  FA8655-06-1-3081 


Even  when  the  results  were  negative,  it  raised  some  interesting  points:  First,  it  seemed  that  the 
visual  depth  had  a  significant  impact  on  user  performance,  but  it  was  a  negative  impact  that  led  to 
mistakes  (Wong  2006).  Again,  the  interface  design  used  in  the  experiment  has  to  be  evaluated.  It 
seemed  that  the  design  choice  was  extremely  biased  towards  the  information  labels.  Therefore, 
only  task  outcomes  directly  arising  from  the  labels  were  bound  to  succeed.  Amaldi  (2006)  pointed 
out  that  visual  displays  have  the  objective  of  attracting  people's  attention  on  the  "relevant" 
information,  but  if  some  information  is  made  too  relevant  the  risk  is  that  the  operators  will  focus 
inappropriately  on  the  information  when  it  is  no  longer  necessary. 

Multiple  Object  Tracking 

Duenser  et  al.  (2006)  replicated  Viswanathan’s  experiment  of  allocation  of  attention  in  different 
depth  planes  using  a  MED  instead  of  a  stereoscopic  display.  They  found  that  using  the  depth 
properties  of  the  MED  increased  users’  performance  when  tracking  multiple  moving  objects  on  a 
computer  screen.  In  the  experiment,  users  could  track  more  objects  correctly  when  they  were 
equally  distributed  over  two  depth  layers. 

These  studies  have  demonstrated  the  potential  that  the  MED  has  above  conventional  single-layered 
displays,  especially  in  search,  monitoring  and  tracking  tasks.  However,  it  is  important  to 
emphasize  that  traditional  design  techniques  have  to  be  reconsidered  when  designing  for  the  MED. 
Its  different  configuration  forces  designers  to  rethink  on  ways  to  keep  the  relationship  between  the 
front  and  the  back  layer  as  well  as  keeping  the  task  compatibility  with  the  suitability  of  the  display 
for  the  desired  perceptual  task  since  different  displays  are  compatible  with  different  tasks  (cf. 
Wong,  Mansour  et  al.  2005) 
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Chapter  4 

Cognitive  Processes  for  the  MLD  and 
Change  Detection 


Using  an  MLD  implies  a  new  set  of  cognitive  analyses  since  the  use  of  perceptual  depth  as  a  cue  to 
enhance  change  detection  has  not  yet  been  investigated.  The  following  sections  provide  an 
overview  of  visual  and  attentional  processes  analyzed  under  the  change  blindness  phenomena  and 
the  use  of  the  MLD.  Concepts  such  as  depth  perception  and  the  deployment  of  attention  in  depth 
are  evaluated. 


Vision  and  Attention  in  the  Eyes  of  Change  Blindness 

Early  theories  of  attention  states  that  our  brains  construct  an  exact  representation  of  what  we  see. 
In  1985,  Feldman  hypothesised  the  existence  of  a  visual  buffer,  a  spatiotopic  memory  store 
believed  to  accumulate  the  contents  of  successive  fixations.  However,  change  blindness  suggests 
that  no  such  buffer  exists.  Irwin  (1996)  stated  that  it  appears  that  the  detailed  contents  of 
successive  presentations  -  including  successive  fixations  -  can  never  be  added,  compared  or 
otherwise  combined  in  their  entirety,  ruling  out  any  large-scale  accumulation  of  information  (cf. 
Rensink  2002).  Noe,  Pessoa  et  al.  (2000)  stated  that  change  blindness  is  not  surprising  because 
"seeing  is  a  temporally  extended  activity  of  visually  exploring  the  environment "(p.  95).  Since  we 
blink  once  or  twice  a  second,  and  saccadic  suppression  occurs  3  or  4  times  a  second,  our  eye 
acquires  visual  information  only  during  these  brief  and  interrupted  windows,  so  they  suggested  that 
it  is  mistaken  to  suppose  that  the  visual  system  has  to  build  up  integrated  world-models  and  it  is 
mistaken  to  suppose  that  vision  in  general  requires  such  models. 

In  fact,  studies  on  monkeys’  brains  conducted  in  1982  raised  interesting  findings  about  the  visual 
system.  Ungerleider  and  Mishkin  found  that  there  were  two  perception  streams  in  the  brain:  the 
dorsal  and  the  ventral  stream.  The  dorsal  stream  said  ‘where’  things  were,  the  ventral  stream  said 
‘what’  things  were.  Thus,  when  people  suffered  dorsal  stream  insult  (stroke  or  brain  injury)  they 
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would  be  able  to  identify  things,  but  not  be  able  to  reach  out  to  them.  When  people  suffered 
ventral  stream  insult  they  could  interact  with  things  but  not  describe  them  to  others. 

In  1991,  Goodale  and  Milner  extended  this  to  ‘how’  and  ‘what’  distinguishing  between  “a  vision 
for  perception  and  a  vision  for  action”  (Goodale  and  Milner  2006,  p.  660).  Their  findings  showed 
that  the  dorsal  stream  picks  up  affordances  requiring  precise  spatial  knowledge  to  prompt  action, 
while  the  ventral  stream  picks  up  representations  and  only  general  spatial  relations  are  necessary. 
The  problem  with  accurate  spatial  knowledge  is  that  it  takes  a  lot  of  space  and  is  only  useful  for 
short  periods  of  time  -  if  a  person  moves  an  inch  to  the  left  then  the  precise  knowledge  is  no  longer 
useful  (Gaukrodger  2007).  Because  the  dorsal  stream  is  concerned  with  action,  not  cognition,  it  is 
expected  people  to  notice  only  those  changes  that  affect  interaction  with  that  specific  object. 

Change  Blindness  studies  have  attempted  to  distinguish  the  neural  correlates  of  change  detection 
from  those  of  change  blindness.  By  using  functional  magnetic  resonance  imaging  (fMRI)  of 
subjects  attempting  to  detect  a  visual  change  occurring  during  a  screen  flicker,  results 
demonstrated  that  although  change  blindness  resulted  in  some  activity,  the  dorsal  activations  were 
clearly  absent  (Beck,  Rees  et  al.  2001;  Pessoa  and  Ungerleider  2004).  These  results  demonstrated 
the  importance  of  parietal  and  dorso-lateral  frontal  activations  for  conscious  detection  of  changes 
in  properties  coded  in  the  ventral  visual  pathway,  and  thus  suggest  a  key  involvement  of  dorsal- 
ventral  interactions  in  visual  awareness  (Beck,  Rees  et  al.  2001) 

If  it  has  been  shown  that  during  visual  disruptions,  the  dorsal  stream  is  not  activated,  and  visual 
disruptions  apparently  swamp  the  visual  transients  away,  then  how  are  we  still  capable  of  detecting 
changes?  Based  on  an  ecological  theory,  we  suggest  that  we  are  able  to  notice  those  changes  that 
that  affect  interaction  with  that  specific  object  that  the  observer  is  interacting  with. 


How  do  we  perceive  in  depth? 


The  ability  for  humans  to  perceive  depth  is  a  complex  neural  activity  which  takes  advantage  of 
physiological  and  environmental  resources,  as  a  means  of  maximising  the  chances  of  producing  the 
least  ambiguous  percept  of  distance  in  a  three-dimensional  space  as  possible  (see  review  by 
Joyekurun  2006).  One  of  the  most  complicated  feats  achieved  by  the  human  brain,  more 
specifically  the  human  visual  system  is  in  the  computation  of  the  third  dimension  of  our  world. 
Also  known  as  the  reverse-optical  problem,  it  arises  from  the  postulate  that  the  human  brain  needs 
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to  infer  all  three-dimensional  qualities  and  quantities  of  a  saccadic-image  from  the  two- 
dimensional  properties  of  a  retinal  projection. 

If  the  spatial  characteristics  of  our  environment  are  computed  from  a  relatively  flat  retinal  image, 
then  it  is  safe  to  assume  that  a  flat  and  synthetic  image  augmented  with  an  appropriate  number  of 
ecologically  valid,  visual  cues  would  be  sufficient  to  provide  any  percept  of  space  and  depth. 
Photographic  and  moving  pictures  sequences  are  a  proof  of  the  process  as  they  capture  a  snapshot 
of  our  environment  but  still  allow  the  human  visual  system  to  infer  an  excellent  number  of  spatial 
percepts  from  them.  The  reproduction  of  such  ecological  cues  has  proved  harder  to  achieve  in 
spatially  flat  presentation  interfaces  and  displays.  The  shortcut  taken  by  many  in  adopting  a  direct 
simulation  of  the  three-dimensional  world  by  reproducing  its  basic  spatial  and  lighting  conditions, 
for  instance  virtual  realities,  has  proved  immensely  useful  in  untangling  many  issues  (Joyekurun 
2006). 

Although  many  have  pointed  out  that  we  do  not  see  ‘depth’  at  all,  Gibson  went  to  the  extreme  to 
claim  that  “there  is  no  such  thing  as  flat  form  perception,  just  as  there  is  no  such  thing  as  depth 
perception”  (1979,  p.  150).  Hence  to  talk  about  ‘depth  perception’  is  possibly  somewhat 
misleading.  What  we  see  is  not  ‘depth’  but  surfaces  and  textures  of  objects.  However,  for 
convenience,  we  will  refer  to  this  phenomenon  as  ‘depth  perception’  and  we  will  explain  a  number 
of  mechanisms  that  come  into  play  to  enable  us  to  perceive  depth. 


The  depth  cues 

Stereopsis 

Stereoscopic  vision  relies  on  two  factors:  binocular  disparity  and  vergence.  Binocular  disparity  is 
the  name  given  to  the  fact  that,  because  our  eyes  are  set  about  6-7cm  apart,  our  retinas  receive 
slightly  different,  but  considerably  overlapping  images  (Lansdown  1996).  Joseph  Harris  in  his 
posthumously  published  Treatise  of  Opticks  (1775)  was  perhaps  the  first  to  suggest  that  the  two 
disparate  images  we  receive  in  binocular  vision  might  be  used  to  produce  depth.  But  it  wasn’t  until 
1832  when  Charles  Wheatstone  showed  that  a  3D  effect  could  be  produced  by  viewing  two- 
dimensional  images  using  his  stereoscope.  The  3D  image  formed  by  2  separate  views  enables  us 
to  estimate  the  depth  of  objects  which  provokes  stereoscopic  cues. 
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Motion  parallax 

The  apparent  relative  motion  of  several  stationary  objects  against  a  background  when  the  observer 
moves  gives  hints  about  their  relative  distance.  This  effect  can  be  seen  clearly  when  driving  in  a 
car,  nearby  things  pass  quickly,  while  far  off  objects  appear  stationary.  Therefore,  moving  the 
head  sideways  or  up  or  down  has  two  effects:  it  provides  a  depth  percept  during  the  motion  from 
the  optic  flow,  and  it  provides  different  points  of  view  (Kooi  2001). 

Gibson  took  the  view  that  it  is  not  just  motion  parallax  but  the  more  general  concept  of  motion 
perspective  that  is  the  primary  factor  in  our  perception  of  depth:  We  perceive  in  order  to  move,  but 
we  must  also  move  in  order  to  perceive  (Gibson  1979,  p.  223).  Motion  perspective  is  the 
continuous  change  in  the  way  objects  — especially  collections  of  objects  —  look  as  an  observer 
moves  about. 

Obviously  detection  of  movement  plays  a  vital  role  in  the  survival  of  animals:  they  must  be  good  at 
perceiving  movement  of  predators  and  of  likely  prey.  Sekuler  (2002)  proposed  that  during 
evolution,  motion  perception  was  probably  shaped  by  selective  pressures  that  were  stronger  and 
more  direct  than  those  shaping  other  aspects  of  vision.  As  a  result  of  such  selective  pressures,  our 
visual  systems  contain  neural  mechanisms  specialised  for  the  analysis  of  motion. 

Texture  gradients 

Gibson  (1979)  claimed  that  a  key  function  of  the  visual  system  is  to  extract  properties  of  the 
surfaces.  He  pointed  out  that  surface  texture  is  one  of  the  fundamental  visual  properties  of  an 
object.  The  texture  of  an  object  helps  us  see  where  an  object  is  and  what  shape  it  has.  On  a  larger 
scale,  the  texture  of  the  ground  plane  on  which  we  walk  produces  a  characteristic  texture  gradient 
that  is  important  in  space  perception.  It  is  difficult  to  design  textured  surfaces  in  computer 
visualizations,  but  in  3D  environments  it  provides  an  essential  cue  to  the  orientation,  shape  and 
spatial  layout  of  a  surface.  Gibson  went  so  far  as  to  claim  that  texture  gradients  were  sufficient  for 
the  guidance  of  ego  motion. 

Overlapping  and  partial  occlusion 

One  of  the  elements  we  use  in  depth  perception  is  our  knowledge  of  the  fact  that  objects  that  are 
nearer  partially  obscure  are  farther  away.  Strong  empirical  findings  have  recently  shown  that 
although  monocular  occlusion  provides  depth  cues,  binocular  occlusion  gives  an  added  feature  in 
the  form  of  metric  depth  (see  review  Joyekurun  2006). 
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Overlaying  is  used  to  convey  an  understanding  of  depth  but  it  is  not  without  its  problems.  Gibson 
suggested  that  the  phenomenon  of  the  superposition  of  objects  is  actually  not  a  clue  to  the  depth  of 
objects  but  a  perception  which  requires  explanation  (1950,  p.l42).  A  man  knows  that  a  near  object 
can  partially  obscure  a  far  object  but  his  retina  does  not,  and  a  retinal  explanation  should  be  sought 
first.  He  proposes  that  the  contours  of  partially  occluded  objects  are  different  from  those  that  are 
not  occluded. 
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Deployment  of  attention  in  depth 


One  of  the  presumed  functions  of  the  visual  system  is  to  facilitate  navigation  within  a  three- 
dimensional  world,  therefore,  one  might  hypothesize  that  attention  is  allocated  within  a 
representation  that  includes  depth  information  (Marrara  and  Moore  2000).  Object-hased  theories 
of  attention  claim  that  selective  attention  can  operate  on  perceptual  objects  and  not  on  the  basis  of 
spatial  location  alone.  They  comprised  2  kinds  of  studies:  those  in  which  two  or  more  objects  are 
displayed  at  the  same  spatial  location  and  those  in  which  the  spatial  location  of  one  or  more  objects 
changes  with  time  (multi-element  tracking).  The  first  kind  of  study  showed  that  in  scenes  with 
superimposed  event  sequences  or  overlapping  shapes,  human  observers  can  selectively  attend  to 
one  of  the  sequences  or  shapes  and  ignore  the  other;  despite  the  physical  overlap  of  spatial  location 
(cf.Viswanathan  and  Mingolla  2002) 

Downing  and  Pinker  (1985)  &  Hoffman  and  Mueler  (1997)  provided  some  of  the  earliest  evidence 
that  attention  can  be  allocated  in  depth.  Studies  have  shown  that  it  is  possible  to  focus  attention  on 
a  particular  depth  plane  defined  by  binocular  disparity  (cf.  Theeuwes,  Atchley  et  al.  1998).  For 
example,  Nakayama  and  Silverman  found  parallel  search  for  a  conjunction  of  colour  and  depth. 
Because  participants  could  direct  attention  to  a  particular  plane  in  depth,  a  target  defined  by  a 
conjunction  of  features  became — within  the  attended  depth  plane — a  target  defined  by  a  single 
feature  (Theeuwes,  Atchley  et  al.  1998). 

Other  studies  have  used  the  Multiple  Object  Tracking  (MOT)  paradigm  to  investigate  conditions 
under  which  depth  may  aid  the  allocation  of  attention  when  the  visual  system  must  simultaneously 
track  a  subset  of  identical  moving  objects.  Pylyshyn  and  Storm  (1988)  conducted  MOT 
experiments  which  suggested  that  the  elements  must  be  tracked  in  parallel  instead  of  performing  a 
serial  process.  Holliday  and  Braddick  (1991)  and  Nakayama  and  Silverman  (1986)  showed  that 
attention  can  be  allocated  to  a  specific  location  defined  by  disparity  where  there  is  no  interference 
from  distracters  in  other  depth  planes,  making  it  easier  to  allocate  attention  in  a  multi-element 
tracking  task  across  two  surfaces  than  within  a  single  surface. 

These  findings  were  corroborated  by  Viswanathan  and  Mingolla  (2002).  Their  experiments  were 
conducted  using  stereoscopic  depth  to  produce  depth  perception.  The  displays  were  viewed 
through  stereo  glasses.  Results  showed  that  both  the  depth  factor  and  the  surface  factor  proved  to 
have  strong  influence  on  performance  in  a  multi-element  tracking  task.  They  demonstrated  that 
“performance  in  a  multi-element  tracking  task  [does]  not  deteriorate  when  attention  must  be 
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allocated  across  two  depth  planes  instead  of  within  a  single  depth  plane,  it  actually  improves” 
(Viswanathan  and  Mingolla  2002,  p.  1427). 

We  have  seen  that  visual  perception  makes  free  use  of  all  cues  available  to  us  as  a  means  of 
allowing  us  to  perceive  in  depth.  Studies  have  demonstrated  that  attention  can  be  allocated  to 
different  depth  planes  without  impairing  performance.  The  next  step  is  to  use  the  lessons  learned 
from  the  studies  about  visual  and  attentional  processes  and  the  use  of  depth  cues  to  propose  a 
representational  design  technique  that  will  allow  utilizing  the  MLD  as  a  tool  to  reduce  Change 
Blindness. 

Depth  perception  in  change  detection  tasks  has  to  be  analyzed  in  a  laboratory  environment  to 
investigate  cognitive  perceptnal  processes  that  have  not  been  investigated  yet.  However  the 
analysis  of  the  relations  of  the  observer  with  the  environment  will  be  taken  into  account  in  further 
research. 
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Chapter  5 

The  5  Effects  of  Change  Blindness 


We  have  detailed  the  various  types  of  perceptual  “hlindness,  the  cognitive  hypothesis  that  have 
attempted  to  explain  it,  and  its  consequences  in  operational  environments.  By  conducting  an 
Emergent  Themes  Analysis,  we  found  some  commonalities  between  twenty  five  studies  conducted 
on  these  types  of  perception-“hlindness”.  These  commonalities  allowed  us  to  systematically 
compare  these  studies  to  find  themes  that  intersect  among  them. 

The  details  of  these  25  studies,  including  an  explanation  of  the  experiments  and  the  authors’  key 
findings  were  inserted  in  a  table  (See  Appendix  1).  Once  this  information  was  in  place,  we  add  an 
additional  column  indicating  our  interpretations  of  those  results.  Once  we  analyzed  the  data  in  this 
table,  we  identified  5  effects,  or  properties,  common  across  the  different  forms  of  change  blindness, 
and  hence  the  likely  effects  that  designers  need  to  consider  when  designing  visual  interfaces  to 
reduce  it. 

These  five  effects  are  (i)  the  effect  of  rate  of  change,  (ii)  the  effect  of  eccentricity,  (iii)  the  effect  of 
proximity,  (iv)  the  effect  of  significance,  and  (v)  the  effect  of  task  relevance. 


Effect  of  the  Rate  of  Change 

An  important  distinction  is  that  between  the  detection  of  dynamic  versus  completed  change.  The 
former  refers  to  “the  perception  of  the  transformation  itself,  [the  latter]  refers  to  the  perception  that 
the  structure  changed  at  some  point”  (Rensink  2002,  p.  249).  Studies  in  which  a  change  is 
presented  at  a  slow  rate,  without  any  visual  disruption,  have  demonstrated  a  high  degree  of  change 
blindness.  Therefore,  not  only  the  visual  disruption  can  mask  the  change  but  also  the  rate  in  which 
the  change  occurs  influences  its  detection  significantly. 
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Simons,  Franconeri  et  al  (2000)  studied  the  effect  of  gradual  changes  in  the  absence  of  visual 
disruptions.  They  found  a  great  degree  of  change  blindness  when  a  change  happens  gradually  over 
a  period  of  time.  They  tested  the  addition  or  deletion  of  objects  and  changes  in  colour  presented 
gradually  versus  the  same  changes  presented  with  the  flicker  technique.  Gradual  changes  in  the 
existence  of  an  object  (addition/deletion)  were  detected  64.3%  of  the  time,  compared  to  57.4% 
when  the  flicker  was  included.  Detection  of  colour  changes  was  better  detected  in  the  flicker 
condition  (41%)  than  in  the  gradual  condition  (31%).  Flowever,  in  both  cases,  a  high  degree  of 
change  blindness  occurred. 


Effect  of  Eccentricity 

Eccentricity  is  the  distance  of  an  element  in  a  visual  display  from  the  foveal  vision.  Elements  in 
the  periphery  usually  go  undetected.  However,  eccentricity  is  not  fixed  to  the  periphery  of  the 
visual  display  but  it  varies  according  to  where  in  a  visual  display  the  observant  is  looking  at. 

Generally  eccentricity  is  confounded  with  relevance  (Podczerwinski,  Wickens  et  al.  2002;  Wickens, 
Muthard  et  al.  2003a).  Eor  instance,  in  the  aviation  domain,  when  pilots  have  to  select  a  flight  path, 
events  that  are  close  to  own-ship  are  more  relevant  to  flight  safety  than  those  located  in  the 
periphery  (Wickens,  Muthard  et  al.  2003a).  Participants  showed  a  greater  accuracy  at  closer 
distances.  Eccentricity  had  a  small,  but  significantly  detrimental  effect  on  detection  speed. 

These  findings  imply  that  surveillance  of  the  display  perimeters  will  deteriorate  and  additional 
methods  should  be  used  to  ensure  that  attention  is  sufficiently  directed  to  these  areas. 

Nikolik  and  Sarter  (2001)investigated  pilot  performance  in  detecting  mode  changes  in  their  flight 
management  system.  They  found  that  pilots  missed  over  10%  of  the  changes  when  using  the 
peripheral  display  that  presented  the  best  performance,  and  that  this  rate  substantially  increased  to 
almost  20%  with  competition  for  visual  attention.  They  showed  that  a  feasible  method  for 
supporting  data-driven  monitoring  and  human-automation  communication  and  coordination  is  the 
use  of  effective  peripheral  visual  cues  and  the  distribution  of  tasks  and  information  across  sensory 
channels. 

Podczerwinski  et  al  (2002)  found  that  eccentricity  had  a  small,  but  significant  effect  on  detection 
speed  (E  =  1.93,  p<.05).  The  limited  magnitude  of  this  effect  is  reflected  in  the  regression  slope  of 
approximately  0.16  sec/degree  of  visual  angle,  or  one  second  of  slowing  for  every  6  degrees  of 
eccentricity.  Participants  showed  a  greater  accuracy  at  closer  distances. 
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In  Mack  and  Rock’s  (1999)  experiment,  in  which  one  of  the  arms  of  a  cross  changed  in  length, 
showed  that  about  25%  of  subjects  were  inattentionally  blind  when  the  cross  was  presented  at 
fixation  and  the  unexpected  object  was  presented  parafoveally  (subjects  typically  detected  the 
critical  stimulus  on  divided-attention  and  full-attention  trials).  They  also  found  that  about  75%  of 
subjects  were  inattentionally  blind  when  the  cross  was  presented  parafoveally  and  the  unexpected 
object  was  presented  at  fixation,  suggesting  an  effortful  shift  of  attention  away  from  fixation  to  the 
cross  and  possible  inhibition  of  processing  at  the  ignored  fixation  location. 


Effect  of  Proximity 

Proximity  refers  to  the  relationship  established  between  elements  in  a  visual  display  determined  by 
its  nearness.  According  to  their  position,  observers  establish  relations  between  them. 

For  instance,  in  the  context  of  the  human-computer  interface,  a  web  site  company  was  analyzed 
due  to  the  low  response  it  had  for  a  specific  training  class.  Exploring  the  Web  site,  they  discovered 
a  striking  flaw  in  the  site’s  interface:  Employees  tried  to  sign  up  but  failed  to  find  the  proper  page. 
The  interesting  thing  about  this  failure  was  that  the  page  was  prominently  advertised  with  a 
colourful  banner  that  was  linked  to  the  sign-up  page.  The  banner  however,  was  not  close  to  other 
link-rich  areas  of  the  display.  Many  employees  had,  indeed,  found  the  page  containing  the  banner 
but  still  failed  to  read  it  so  they  could  follow  the  link.  This  failure  was  replicated  in  the  lab  and 
dubbed  it  “Banner  Blindness”  because  participants  appeared  to  be  completely  unaware  of  a 
prominent  signal  that  was  directly  related  to  their  current  goals  (Varakin,  Levin  et  al.  2004). 

This  experiment  showed  that  users  have  expectations  of  where  in  the  display  to  look  and  may 
ignore  most  of  what  is  visible  at  any  given  time. 


Effect  of  Significance 

Change  detection  is  definitely  influenced  by  meaning  and  significance  of  the  elements  shown  in  a 
picture  or  any  visual  interface.  Observers  categorize  elements  in  a  scene  according  by  judging  how 
elements  convey  meaning  to  it. 

Rensink  et  al  (1997)  classified  objects  in  a  scene  as  of  central  or  marginal  interest  according  to  the 
degree  of  interest  in  the  part  of  the  scene  being  changed  (obtained  from  independent  judges  in  a 
prior  pilot  experiment).  Central  interest  changes  were  most  often  detected  as  soon  as  they  occurred 
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while  marginal  interest  changes  were  generally  seen  after  some  occurrences.  This  classification, 
although  named  in  a  different  way,  is  seen  repeatedly  in  several  experiments,  if  participants  feel 
more  familiar  with  the  stimuli  changing,  whether  is  a  person  or  an  object,  they  will  detect  the 
changes  faster  than  if  they  do  not  find  any  meaning  in  the  changed  stimuli. 

According  to  Rensink  (2000),  if  a  change  affects  the  meaning  of  a  scene,  observers  will  be  more 
likely  to  compare  the  details  of  the  scene  to  their  representations  and  thereby  detect  the  change. 
For  instance,  Richard  et  al.  (2002)  found  that  detection  of  changes  in  driving  scenes  was 
significantly  faster  for  driving-relevant  than  driving-irrelevant  changes  (cf.  Durlach  2004). 
Rensink,  O’  Regan  et  al.  (1997)  concluded  that  change  detection  depended  greatly  on  the 
significance  of  the  part  of  the  scene  being  changed,  with  fastest  identification  for  those  structures 
of  greatest  interest  (p.  8). 

Studies  in  Fluman  Factors  have  emphasized  the  importance  of  meaning  in  relation  with  the  activity 
participants  are  performing.  In  most  laboratory  experiments  participants  are  usually  passively 
looking  at  a  screen.  However,  a  study  conducted  in  the  London  Underground  (London  railway 
public  transport  network)  to  develop  technologies  that  support  ‘awareness’,  showed  that  station 
supervisors  and  other  control  room  personnel,  do  not  passively  monitor  the  displays  waiting  for 
‘something  to  happen’  or  just  glance  at  the  screens  to  gain  a  general  ‘awareness’  of  the  activities 
occurring  within  the  station,  but  actively  undertake  analysis  of  the  conduct  they  see  on  the  screens 
and  through  the  window  of  the  control  room  (Luff,  Heath  et  al.  2006). 

Simons  and  Levin  (1997)  examined  the  effect  of  familiarity  in  a  variation  of  the  “changing 
experimenter”  scenario  described  in  Chapter  1.  If  the  experimenter  was  dressed  as  a  student  and 
he  asked  other  students  for  directions  (in-group  effect),  the  change  of  conversation  partner  was 
noticed  almost  100%  of  the  time.  However,  if  the  experimenter  was  dressed  as  a  construction 
worker  (out-group),  students  noticed  the  change  only  35%  of  the  time.  This  in-group/out-group 
effect  relates  very  closely  to  central  and  marginal  interests  analysis.  When  participants  did  not  feel 
familiar  to  the  person  they  were  talking  to,  they  did  not  notice  the  change  of  conversation-partner. 
Their  main  task  was  to  provide  directions. 

Archambault  et  al.  (1999)  analyzed  a  “categorization”  effect.  For  their  experiments,  participants 
were  presented  with  a  picture  of  an  office  that  had  a  desk,  a  computer  monitor,  a  coffee  mug, 
papers,  books,  etc.  Using  the  flicker  technique,  changes  were  made  only  to  the  computer  or  to  the 
coffee  mug.  Participants  had  to  learn  these  objects  in  a  specific  level  as  Mary's  mug  or  Peter's 
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computer,  or  in  a  general  level  (computer  or  mug).  Participants  were  divided  into  2  groups;  one 
will  specifically  learn  computers  in  a  specific  level  and  the  mugs  in  a  general  one,  the  second 
group  will  do  the  opposite. 

At  a  specific  level,  participants  noticed  the  change  faster  than  if  they  learned  it  at  a  general  level. 
Archambault  et  al.  conducted  a  second  experiment  in  which  each  group  will  learn  some  coffee 
mugs  and  some  computers  at  a  general  level  and  some  at  a  specific  level.  Again,  faster  detection 
occurred  if  objects  were  learned  at  a  specific  level  than  if  they  were  learned  at  a  general  level  (2.43 
alternations  vs.  4.32).  Therefore,  level  of  categorization  at  which  an  object  is  learned  can  affect  its 
visnal  encoding  and  perception 

On  another  set  of  experiments.  Mack  and  Rock  (1999)  found  that  while  the  inattentional  blindness 
effect  was  robust  for  meaningless  stimuli,  their  participants  almost  invariably  noticed  when  their 
own  names  were  presented.  According  to  them,  there  were  a  few  other  stimuli  that  captured 
attention  under  conditions  of  inattention  such  as  a  cartoon-like  happy  face  due  its  high  signal  value 
and  a  high  degree  of  familiarity.  However,  if  the  participant’s  name  was  slightly  misspelt  or  a 
scrambled  or  sad  version  of  the  face  was  presented,  inattentional  blindness  returned. 


Effect  of  Relevance  of  the  Task 

Some  have  argued  that  the  procedures  for  change  blindness  experiments,  like  changes  to  scenes 
lacked  clarity;  the  task  was  relatively  vague,  asking  observers  to  search  for  arbitrary  changes  to 
pictures  with  no  idea  of  the  magnitude  or  content  of  the  change  (DiVita,  Nugent  et  al.  2004). 
Consequently,  Change  Blindness  was  stndied  with  operational-domain  tasks,  and  it  was  shown  that 
change  detection  improved  when  the  changes  were  meaningful  to  the  task. 

The  importance  of  meaning  and  context  has  also  been  studied  in  other  areas  such  as  the  theory  of 
activity  that  affirms  that  there  is  a  dependent  relationship  between  a  computer-based  system  and 
the  practice  in  which  it  is  used,  in  fact,  Chaiklin  states  that  while  the  goal  of  the  task  is  important, 
its  meaning  can  only  be  understood  in  relation  to  its  function  in  the  activity  (2007). 

Podczerwinski  et  al.  (2002)  compared  highlighting  benefits  and  de-cluttering  costs  of  electronic 
map  design  in  dynamic  settings  and  examined  the  potential  cost  of  change  blindness  phenomenon 
in  de-cluttering  conditions.  They  demonstrated  that  detection  of  relevant  changes  (changes  that 
cause  a  potential  conflict)  in  both  traffic  and  weather  systems  were  superior  to  that  of  irrelevant 
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changes.  Change  detection  was  only  at  50%  accuracy,  but  improved  by  changes  that  were  more 
salient  (spatially,  not  digitally  represented),  and  were  more  meaningful  (causing  a  conflict  with  the 
flight  path).  Changes  that  were  relevant  to  the  pilot’s  flight  planning  task  were  detected  nearly 
four  seconds  faster  than  irrelevant  changes.  Relevant  conflict  changes  were  detected  75%  within 
an  average  response  time  of  12  sec,  whereas  non-relevant  changes  were  detected  35%  with  an 
average  response  time  of  18  sec. 

One  of  the  attempts  to  study  whether  individual  characteristic  of  the  observer  selectively  influence 
the  ability  to  detect  changes  is  Werner  and  Thies’  study  (2000)  on  expertise  in  American  football 
domain.  An  implication  is  that  experts  should  be  superior  in  detecting  changes  in  domain-relevant 
material,  compared  to  novices.  Participants  had  to  detect  changes  in  American  football  scenes 
(semantic  changes);  their  performance  was  compared  to  scenes  that  showed  non-relevant  changes 
to  the  domain  like  traffic-related  information.  Results  showed  that  American  football  experts 
detected  changes  about  the  game  faster  than  novices  did.  “Novices  failed  to  detect  changes  in 
football  images  twice  as  often  as  experts  (17.6%  vs.  8.3%,  111%  difference).  [However],  for  non 
semantic  changes,  the  difference  between  novices  and  experts  was  22%  (42.1%  vs.  35%)”  (p.  172). 
These  results  demonstrated  that  expertise  in  a  specific  domain  appear  both  to  generally  ease 
detection  of  changes  for  images  from  that  domain  as  well  as  selectively  increase  observers’ 
sensitivity  to  domain-related  semantic  changes  in  those  images. 


These  commonalities  between  studies  have  provided  us  with  a  framework  to  understand  change 
blindness  and  present  some  properties  of  the  change  blindness  phenomenon  that  designers  should 
pay  attention  to  when  designing  visual  interfaces,  especially  for  complex  environments 
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Chapter  6 

Discussion  and  Conclusions 


In  this  paper  we  reported  on  part  of  a  study  to  understand  how  information  layering  techniques  can 
be  used  to  reduce  change  blindness.  Change  blindness,  in  general,  is  the  failure  of  the  human  to 
detect  changes  to  information  that  occur  within  his  or  her  visual  field.  There  are  several  variants  of 
perception-  “blindness”  known  as:  change  blindness,  inattentional  blindness,  comparison  blindness, 
and  repetition  blindness.  Failure  or  delays  in  observing  changes  in  complex,  dynamic  and  safety 
critical  systems  by,  say,  operators  can  lead  to  disastrous  outcomes.  Hence,  drawing  together  the 
different  reasons  reported  in  the  literature  can  provide  us  with  a  first  step  towards  developing 
design  techniques  for  reducing  change  blindness  in  the  hope  that  we  can  contribute  to  better  and 
safer  systems. 

Change  and  other  perception-“blindness”  might  occur  for  a  number  of  reasons:  Many  authors 
interpreted  these  ‘blindness’  as  a  failure  of  visual  awareness,  or  as  a  failure  of  internal  visual 
representation.  Others  argued  that  is  not  just  a  lack  of  visual  representations  but  suggest  that  we 
need  focus  attention  to  detect  changes  (Rensink,  O'Regan  et  al.  1997),  while  others  suggest  that 
attention  might  not  be  the  bottleneck  on  what  is  encoded  and  represented  in  our  brain,  but  perhaps 
memory  or  other  “access  limitations”  provide  a  bottleneck  on  what  information  can  be  used  in 
making  reports  about  the  detection  of  changes  (Noe,  Pessoa  et  al.  2000).  Others  agree  that 
attention  might  be  required  to  consciously  experience  a  change,  but  it  is  not  necessary  since 
unaware  detection  of  change  influence  performance(Fernandez-Duque  and  Thornton  2000). 

Whether  we  store  internal  representations  of  what  we  perceive  or  the  world  serves  as  a  visual  store, 
is  still  debatable.  If,  on  the  other  hand,  change  blindness  is  not  a  problem  of  internal  visual 
representations  but  of  attention,  then  how  is  attention  directed  in  the  first  place?  Although  bottom- 
up  cues  capture  attention,  they  may  be  influenced  by  top-down  attentional  settings,  for  instance 
expectations  of  where  the  target  will  appear  (Vecera  and  Rizzo  2003).  Then,  do  goals, 
expectations  and  intentions  of  the  observer  influence  how  attention  is  deployed? 
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We  propose  that  the  explanations  for  these  phenomena  may  not  lie  in  the  internal  representations 
of  the  brain  but  in  the  relationship  of  the  observer  and  his/her  environment,  in  the  meaning  and 
context  of  the  task  they  perform  as  well  as  the  way  elements  are  represented  and  positioned  in  the 
visual  display. 

Based  on  an  Emergent  Themes  Analysis  approach,  we  identified  5  effects,  or  properties,  common 
across  the  different  forms  of  change  blindness,  and  hence  the  likely  effects  that  designers  need  to 
consider  when  designing  visual  interfaces  to  reduce  change  blindness.  These  five  effects  are  (i)  the 
effect  of  rate  of  change,  (ii)  the  effect  of  eccentricity,  (iii)  the  effect  of  proximity,  (iv)  the  effect  of 
significance,  and  (v)  the  effect  of  task  relevance. 

The  first  three  effects  are  related  to  the  way  elements  in  a  visual  display  are  represented.  The 
studies  mentioned  above  have  shown  that  eccentricity,  rate  of  change  and  position  of  the  elements 
might  enhance  or  deteriorate  the  accuracy  and  response  times  of  observers  in  change  detection 
tasks.  Other  factors  such  as  the  size  of  the  computer  screen  influence  change  detection 
performance  negatively,  but  not  significantly,  probably  because  for  small  displays,  the 
minimisation  of  the  display  elements  decreased  legibility  and  resolution.  Eccentricity  of  the 
elements  in  the  display  was  confounded  with  relevance.  Eor  instance,  in  the  aviation  domain, 
events  (changes)  closer  to  own-ship  generally  are  more  relevant  to  flight  safety.  Saliency  of  the 
elements  in  the  display,  especially  the  use  of  different  intensities  increased  detection  accuracy  for 
the  highlighted  elements  but  affect  the  detection  for  those  low-lighted.  The  speed  at  which  a 
change  occurs  also  affects  detection.  Changes  that  occur  gradually  might  go  unnoticed.  Finally, 
position  of  the  elements  in  a  visual  display  informs  observers  about  the  relationship  between  them, 
and  therefore  proximity  of  related  items  or  links  is  very  important. 

The  last  two  effects,  effects  of  significance  and  relevance  of  the  task  are  more  related  with 
meaning  and  context.  It  has  been  shown  that  humans  are  really  bad  at  detecting  random  changes. 
Observers  pay  more  attention  to  elements  that  convey  a  meaning,  say,  to  a  scene;  or  are  more  able 
to  detect  those  changes  that  are  relevant  to  their  task  or  conflict  with  it.  Observers  detect  changes 
that  provide  them  with  a  meaning  that  can  only  be  understood  in  relation  to  its  function  in  the 
activity. 

The  use  of  different  techniques  such  as  blinking,  use  of  various  intensities,  reverse  video  and 
boxing  have  proven  unsuccessful.  We  hypothesised  that  perceptual  depth  can  be  used  as  a  cue  to 
grab  attention.  Studies  conducted  in  a  new  Multi-Layered  Display  (MED)  technology  have  shown 
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that  this  display  have  potential  to  present  increasing  amounts  of  data  in  one  computer  display  due 
to  its  physical  depth  supporting  the  allocation  of  attention  in  depth,  conjunction  searches  if  depth  is 
combined  with  other  cues  such  as  colour,  and  faster  reading  times  in  conditions  that  have  lead  to 
poor  legibility  on  conventional  displays. 

Further  research  is  required  to  investigate  the  influence  of  perceptual  depth  for  change  detection 
tasks.  During  the  completion  of  this  review,  some  issues  that  have  been  studied  in  other  areas  such 
as  scene  analysis  for  artificial  intelligence  and  experiments  on  Just  Noticeable  Differences  “jnd” 
seem  to  offer  some  plausible  explanations  and  suggestions  for  future  analysis. 

Change  Blindness  and  Inattentional  Blindness  experiments  provided  simple  procedures  to  study 
cognitive  processes  involve  in  change  detection,  however  they  also  provided  high  levels  of 
ecological  validity  for  questions  concerning  human-computer  interaction.  From  now  on,  when 
designing  interfaces,  it  is  not  enough  to  evaluate  if  the  elements  in  the  visual  display  are  relevant 
but  we  have  to  start  questioning  if  the  elements  in  the  visual  display  are  seen  at  all. 
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AUTHORS 

TYPE  OF 

STUDY/TECHNIQUE 

EXPERIMENT 

DESCRIPTION 

KEY  FINDINGS  CONCLUSIONS 

OUR 

INTERPRETATIONS 
(Properties  of  Failures 
of  Visual  Awareness) 

RENSINK, 
O'REGAN  AND 
CLARK  1997 

Change  Blindness  / 
flicker 

EXPERIMENT  1:  flicker 

sequences  were  usually 

composed  of  an  original  image 
A  and  modified  version  A' 
displayed  in  the  sequence  A,  A, 
A',  A',  with  grey  blank  fields 
placed  between  successive 
images.  Each  image  was 
displayed  for  240  ms  and  each 
blank  for  80  ms.  Changes 

were  further  divided  according 
to  the  degree  of  interest  in  the 
part  of  the  scene  being 
changed:  Central  vs.  Marginal 
Interests  (Cl  vs.  Ml). 

EXPERIMENT  2:  A,  A',  A,  A' 
presenting  each  image  for  560 
ms.  EXPERIMENT  3:  =exp  1 
with  cues.  Partially-valid 

condition,  cues  were  divided 
equally  into  valid  cues  (naming 
the  part  of 

the  scene  changed)  and  invalid 
cues  (naming  some  other  part). 
In  the  completely-vaiid 

condition,  cues  were  always 
valid. 

EXPERIMENT  1:  A,  A,  A',  A' . Ml:  17.1 

alternations  (10.9  sec)  to  identify  change.  Cl: 

7.3  alternations  (4.7  sec).  Without  flicker 

1.4  alternations  (0.9  sec)  to  detect  both  Ml 

and  Cl  changes.  EXPERIMENT  2:  A,  A',  A, 
A’  -  same  effect.  EXPERIMENT  3:  valid  cues 
always  caused  identification  of  both  Ml  and 
Cl  changes  to  be  greatly  sped  up.  For 
completely-valid  cues,  the  difference  in 
response  times  for  Mis  and  CIs  declined  to 
the  point 

where  it  was  no  longer  significant. 

Central  interests  (CIs)  were 
defined  as  objects  or  areas 
mentioned  by  three  or  more 
observers,  marginal 

interests  (Mis)  were  objects 
or  areas  mentioned  by 
none.  Changes  that  are 
defined  as  of  CENTRAL 
INTEREST  are  detected 
faster  than  marginal  ones. 
DELETION  OF  OBJECTS 
IS  DETECTED  EASIER 
THAN  ADDITIONS. 

Elements  defined  as  of 
central  interest  provide 
meaning  to  the  scene 

1 


RENSINK, 
O'REGAN  AND 
CLARK  1999 


Change 

Blindness/Mud-splash 


CB  can  occur  even  when  the 
disruption  does  not  cover  or 
obscure  the  changes.  The 
authors  used  48  pairs  of 
pictures,  consisting  of  an 
original  and  a  modified  picture 
(each  displayed  for  3  sec), 
each  pair  presented  cyclically 
with  an  80-ms  duration  "mud 
splash"  superimposed  at  the 
moment  of  the  change.  There 
was  no  disruption  in  visual 
continuity  at  the  moment  of  the 
change.  10  observers  were 
asked  to  press  a  button  as 
soon  as  they  identified  the 
change,  which  could  have  been 
a  large  object  or  region  of  the 
picture  shifting  in  location, 
changing  colour,  or  appearing 
or  disappearing.  The  change 
could  be  either  a  "central 
interest"  or  a  "marginal  interest" 
element. 
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Results  show  that  central  interest  changes 
were  usually  detected  as  soon  as  they 
occurred,  whereas  marginal-interest  changes 
were  seen  only  on  their  2nd  or  later 

occurrences.  Findings  suggest  that  CB  Central  vs.  marginal  interest 
occurs  because  the  internal  representation  of  =  participants  detect 
the  visual  world  is  rather  sparse  and  meaning  from  elements  of  a 

essentially  contains  only  central-interest  scene 

information.  Only  the  parts  of  the 

environment  that  observers  attend  to  and 
encode  as  "interesting"  are  available  for 

making  comparisons. 
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MACK  AND 
ROCK  1998 


Inattentional  Blindness 


A  small  cross  was  shown 
briefly  on  a  computer  screen  for 
each  of  several  experimental 
trials  and  asked  participants  to 
judge  which  arm  of  the  cross 
was  longer.  After  several  trials, 
an  unexpected  object,  such  as  a 
brightly  coloured  rectangle, 
appeared  on  the  screen  along 
with  the  cross. 


flexible  selection  theory  of  attention: 
depending  on  the  nature  of  the  stimuli  (high- 
or  low-level  attributes).  System  operates  to 
minimize  effort  and  so  will  select  on  the 
basis  of  a  low-level  attribute  like  size  if 
possible  but,  if  not,  will  process  the  input 
more  deeply,  as  seems  to  be  the  case  with 
lexical  stimuli .  If  the  critical  stimulus  has  no 
particular  intrinsic  signal  value  and  is 
irrelevant  to  the  subject's  assigned  task,  the 
attentional  warrant  that  permits  a  stimulus  to 
pass  from  implicit  to  explicit  perception  is 
minimal,  so  that  only  its  presence  or  its  bare 
bone  features  are  perceived.  If  the  same 
stimulus  were  to  fall  outside  the  zone  of 
attention,  then  it  is  far  more  likely  that  even 
its  presence  will  go  undetected,  because  it 
does  not  have  the  benefit  of  even  minimal 
attentional  processing.  Drew  a  distinction 
between  conscious  perception  and  implicit 
perception.  The  term  'perceive'  (or  'notice'  or 
'see')  mean  that  observers  have  at  some  point 
had  a  conscious  experience  of  an  object  or 
event,  it  is  important  to  note  that  even  when 
observers  do  not  perceive  an  object,  it  may 
still  have  an  implicit  influence  on  their 
subsequent  decisions  and  performance 


Meaningfulness  and 

relevance  affect  the 
detection  of  unexpected 
events. 
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(i)  About  25%  of  subjects  are  inattentionally 

blind  when  the  cross  is  presented  at  fixation 
and  the  unexpected  object  is  presented 
parafoveally  (subjects 

typically  detect  the  critical  stimulus  on 
divided-attention  and  full-attention  trials). 

(ii)  About  75%  of  subjects  are  inattentionally 
blind  when  the  cross  is  presented 
parafoveally  and  the  unexpected  object  is 
presented  at  fixation,  suggesting  an  effortful 
shift  of  attention  away  from  fixation  to  the 
cross  and  possible  inhibition  of  processing  at 
the 

ignored  fixation  location,  (iii)  These  levels  of 
detection  are  no  different  for  features  thought 
to  be  preattentively  processed  (e.g.  colour, 
orientation,  motion)  and  those  thought 
to  require  effort,  (iv)  Although  objects 
composed  of  simple  visual  features  are  not 
easily  detected,  some  meaningful  stimuli  are. 
Observers  typically  notice  their  own  name  or 
a 

smiley  face  even  when  they  did  not  expect  it. 

SIMON  AND 

CHABRIS  1999 

Inattentional  Blindness 

75sec  video  of  two  teams  of  3 
people  each,  wearing  a  white  or 
a  black  T-shirt  respectively.  At 
44-48  sec  an  unexpected 
event,  either  a  woman  in  a 
gorilla  costume  or  a  woman 
with  an  open  umbrella  walked 
through  the  scene.  Participants 
were  asked  to  make  silent 
count  of  the  passes  total 
number  of  passes  for  the  easy 
condition  or  bounces  and  aerial 

Out  of  all  192  observers  across  all  conditions, 
46%  failed  to  notice  the  unexpected  event. 
More  observers  noticed  the  unexpected  event 
in  the  Opaque  condition  (67%)  than  in  the 
Transparent  condition  (42%).  More 

observers  noticed  the  unexpected  event  in 
the  Easy  (64%)  than  in  the  Hard  (45%) 
condition.  The  Umbrella  Woman  was 

noticed  more  often  than  the  Gorilla  overall 
(65%  vs.  44%).  Observers  are  more  likely  to 
notice  unexpected  events  if  these  events  are 
visually 

4 


passes  for  the  hard  condition. 


An  experimenter  approached  a 
pedestrian  to  ask  for  directions. 
While  the  pedestrian  was 
providing  directions,  two 
additional  experimenters, 
carrying  a  door,  passed 
between  the  initial  experimenter 
and  the  pedestrian.  During  this 
brief  interruption,  a  different 
person  replaced  the  original 
experimenter.  Exp  2:  in 

SIMON  AND  Change  Blindness  group/out  group  effect. 

LEVIN  1998  Real- world  occlusion  Experimenter  dressed  as  a 

student  or  as  a  construction 
worker.  EXP3:  a  female 
experimenter  dressed  in 

athletic  clothing  and  carrying  a 
basketball  approaches  a 
passed  by  in  public  and  asks 
directions  to  a  gym.  During  this 
interaction,  a  crowd  of 
confederates  walked  between 
the  two  and  surreptitiously  took 
the  basketball  away. 
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similar  to  the  events  they  are  paying  attention 
to.  (On  the  basis  of  our  results  it  is  logically 
possible  that  dissimilarity  to  the  ignored 
events  is  instead  the  crucial  factor.)  Objects 
can  pass  through  the  spatial  extent  of 
attentional  focus  (and  the  fovea)  and  still  not 
be  'seen'  if  they  are  not  specifically  being 
attended. 


Even  though  the  two  experimenters  looked 
quite  different  and  had  distinctly  different 
voices,  approximately  50%  of  the  subjects 
failed  to  notice  that  they  were  talking  to  a 
different  person  after  the  door  passed. 
EXPERIMENT2:  Categorization  -  direction 
asker  and  direction-giver  students  almost 
100%  detection,  d-asker  =  construction 
worker  &  d-giver  =  student  35%  detection. 
EXP3:  When  asked  if  they  noticed  anything 
changed  or  anything  different  about  her 
appearance,  a  minority  of  observers  reported 
noticing  that  the  basketball  was  gone.  But 
when  asked  a  follow-up  question  specifically 
referring  to  the  basketball,  most  of  the 
remaining  observers  'remembered'  the 
basketball  and  were  able  to  describe  its 
unusual  colouring.  Thus,  a  visual  change  can 
be  encoded  but  not  explicitly  reported  until  a 
specific  retrieval  cue  is  provided. 


IN  GROUP/OUT  GROUP 
EFFECT.  IMPLICIT 

PERCEPTION 
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Podczerwinski 
Wickens 
Alexander  2002 


Change  Blindness  - 
Out  of  sight  out  of  mind 
OOSOOM/Flight 
simulation  highlighting 


Eight  trials  using  4  display 
formats  baseline,  traffic 
highlighted,  traffic  low  lighted, 
and  manual  de-cluttering 
displays.  They  answered 
focused  and  divided  attention 
questions,  detected  changes  in 
the  trajectory  of  traffic  and 
weather,  and  avoided  conflict 
events  by  manoeuvring 
vertically. 


Overall  only  38%  of  changes  were  detected. 
The  heading  changes  40%  detection. 
Changes  in  digital  altitude  -12%  detection. 
Heading  changes  (more  salient  ones)  were 
poorly  detected  in  the  manual  de-cluttering 
condition  supporting  the  OOSOOM.  Tracking 
error  was  best  in  the  traffic  highlighted 
condition...  A  highlighting  a  display  element 
increased  the  quality  (speed  and  accuracy)  of 
detecting  its  change.  Relevant  conflict 
changes  were  detected  75%  RT  12  sec  on 
average.  Non  relevant  changes  were 
detected  35%  RT  18  sec.  With  relevance 
removed,  distance  (eccentricity)  had  a  small, 
but  significant  effect  on  detection  speed  (F  = 
1.93,  p<.05).  The  limited  magnitude  of  this 
effect  is  reflected  in  the  regression  slope  of 
approximately  0.16  sec/degree  of  visual 
angle,  or  one  second  of  slowing  for  every  6 
degrees  of  eccentricity.  Greater  accuracy  at 
closer  distance.  The  concept  of  “display 
proximity”  as  defined  by  intensity  differences 
does  not  appear  to  behave  in  the  same 
manner  as  when  proximity  is  defined  by 
spatial  differences. 


RELEVANT  CHANGES 
(those  that  can  cause 
conflict)  ARE  DETECTED 
FASTER  THAN  NO 
RELEVANT.  PGP  behave 
different  behaviour  when 
using  different  intensities 
than  when  using  spatial 
differences 
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WICKENS  C, 

MUTHARD  E,  CB/intensity 
ALEXANDER  A  eccentricity 
2003 


and 


Pilots  were  asked  to  detect 
changes  in  the  movement  or 
altitude  of  weather  systems  or 
traffic  aircraft,  which  were 
represented  in  integrated 
hazard  displays.  Experiment  1: 
change  detection  as  a  function 
of  the  distance  of  the  event 
from  own-ship  and  the 
presence  of  the  event  in  a 
highlighted  or  low  lighted 
hazard  domain.  Experiments  2 
and  3:  role  of  display  size  and 
event  eccentricity.  Integrated 
hazard  displays  depicted 
mountainous  terrain  ground 
map,  weather  and  traffic 
domain 


Analyses  revealed  that  change  detection 
was  superior  for  events  that  were  in  the 
highlighted  display  database  and  that 
performance  was  slightly  degraded  for  more 
eccentric  events.  It  was  hypothesised  that 
display  size  would  Impact  distance 
judgements.  DISPLAY  SIZE  WAS  NOT 
SIGNIFICANT  BUT  IT  CAN  DEGRADE 
PERFORMANCE  IN  SMALL  DISPLAYS  DUE 
TO  THE  MINIFICATION  OF  THE  ELEMENTS 
WHICH  DECREASE  LEGIBILITY  AND 
RESOLUTION.  Changes  within  the  nearby 
field  of  view  are  more  likely  to  be  detected 


Eccentricity  =  changes  that 
occurred  nearby  were 
easier  to  detect 
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WICKENS  C, 

MUTHARD  E 

2003 

CB/intensity  and 

eccentricity 

After  selecting  a  flight  path, 
pilots  were  asked  to 

monitor  the  safety  of  the  plan 
by  seeking  and 

detecting  changes  in  the 
altitude  or  trajectory  of 
weather  systems  and  traffic 
aircraft.  Pilots  were 

instructed  to  press  a  key  when 
they  detected  a  change 
and  inform  the  experimenter  of 
the  item  that  changed 
and  the  nature  of  the  change. 
In  one  fourth  of  trials,  a 
change  substantially  threatened 
the  safety  of  the 

flight  path.  At  the  midpoint  of 
each  trial,  participants 

were  asked  to  recommend  the 
safer  plan  to  a 

following  aircraft  that  would  be 
confronted  with  the 

same  flight  path  choice  as  the 
participant,  while 

taking  into  account  the  changes 
that  had  occurred  up 

to  that  point. 

With  an  automation  aid,  accuracy  and 
confidence  was  higher  than  without  (accuracy 
high  workload=78.1%vs  control=65.6%. 
Speed  accuracy  trade  off:  changes  in  the  low 
workload  condition  were  detected  36%  more 
accurate  but  6.7  sec  slower  than  high- 
workload  condition.  WITH  IMPERFECT 

AUTOMATION  60.9%  OF  MISSES,  NO 
AUTOMATION  AID  39.1%  MISSES 

TRADE  OFF  OF 

HIGHLIGHTING 

WERNER  AND 
TRIES  2000 

Change  Blindness 

Expertise  Domain 

30  PAIRS  OF  COLOUR 
IMAGES.  Single  modified 
element.  Categoryl :  action 
football  scenes  few  players  and 
a  referee.  Category  2:  bird's 
eye  view  of  playing  formations. 
Category  3:  traffic-related 
scenes.  2  type  of  changes: 
semantic  -  alter  an  item  that 
was  important  for  the 
interpretation  of  the  scene,  and 

Experts  have  a  distinct  advantage  over 
novices  for  detecting  semantic  changes  in 
images  from  their  domain,  are  faster  on 
domain-related  images.  Significant 

interaction  between  expertise  and  type  of 
change  detected.  Significant  difference 

between  change-type:  semantic  changes 
were  easier  to  detect  than  non-semantic.  . 
For  category  3  no  significant  interaction  found 
except  that  semantic  changes  were  detected 
slower  than  non-semantic 

RELEVANCE  OF  THE 
CONTEXT  MAKE  CHANGE 
DETECTION  FASTER  = 
meaning  based  on  expertise 
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non-semantic  -  irrelevant  parts 
of  the  scene 

KANWISHER 

1996 

Repetition  Blindness 
(RB) 

0  selective  forgetting  of 
repeated  items  but  failure  to 
assign  a  distinct  episodic 
representation  to  the  repeated 
item 

RB  is  not  due  to  selective  forgetting  repeated 
items  but  a  failure  to  assign  a  distinct 
episodic  representation  to  the  repeated  item 

RB  effects  are  observed 
with  simultaneously 

presented  letters  (LIKE  IN 
COMPARISON 

BLINDNESS) 

DiVITA  AND 

NUGENT  2004 

Change  Blindness  in 
Naval  Combat 

Displays/  2  monitors 

4  Categories  were  tested: 
Course,  speed,  range  and 
bearing  in  a  simulated  CIC 
(Combat  Information  Centre) 
console.  20  critical  changes:  14 
tactically  significant  attributes  & 

6  different  new  contacts 
entering  the  scene. 

It  is  more  difficult  to  detect  changes  without 
the  graphical  display  (map/air  corridors)  at 
the  background.  "IF  THE  OPERATOR'S 
SITUATIONAL  AWARENESS  IS  POOR,  HE 
OR  SHE  MAY  FAIL  TO  OBSERVE  SOME  OF 
THESE  RELATIONSHIPS,  WHICH  IN  TURN 
LEADS  TO  A  FAILURE  IN  CHANGE 
DETECTION"  p.  216 

ANCHORS  OR  GRAPHICS 
THAT  ACT  AS  A 

REFERENCE  SHOULD  BE 
KEPT  ALWAYS  ON  SIGHT 
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2  monitors  placed  in  front  of  the 
participant.  Pilots  were 

required  to  detect  and  identify 
24  experimenter-induced  mode 
transitions  and  24  other 
scenario  events.  The  mode 
transition  detection  task 
consisted  of  pressing  one  of 
two  buttons  on  the  side-stick  as 
soon  as  a  mode  transition  was 
detected.  In  case  of  other 
scenario  events,  pilots  were 
required  to  indicate  orally  the 
observation  and  nature  of  the 
event.  Pilots  were  randomly 
assigned  to  one  of  three  types 
of  mode  transition  displays: 
PERIPHERAL  (baseline  FMA,  enhanced  FMA, 

NIKOLIC  AND  CHANGE  or  ambient  strip). 

CARTER  2001  DETECTION/  Flight  (l)BASELINE:  Transitions 

simulation  technique  between  modes  are  signalled 

by  a  thin  green  rectangular 
outline  box  drawn  around  the 
new  mode  for  10  s.  The  box 
subtends  2.5°  VA.  (2) 

Enhanced  FMA  signalled  a 
mode  transition  by  the  onset 
and  10-s  presentation  of  a  light. 
The  enhanced  FMA  was 
shown  in  the  same  location  as 
the  current  FMA,  a  solid  yellow 
box  for  a  thrust  mode  transition 
(left  column)  and  a  blue  box  for 
a  heading  mode  transition 
(middle  column).  (3)The 
ambient  strip  signalled  mode 
transitions  by  the  onset  of  a 
_  thin,  colour  band,  using  the 
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Enhanced  FMA  and  ambient  strip  support 
faster  detection  as  compared  with  the 
baseline  FMA.  The  relatively  low  detection 
performance 

in  the  baseline  condition  replicates  pilots’ 
difficulties  with  maintaining  mode  awareness 

on  modern  flight  decks  and  can  be  explained  Abrupt  onsets  per  se  may 

by  the  current  FMA’s  low  salience  relative  to  not  be  sufficient  to  capture 

its  visually  saturated  surround  on  the  attention,  especially  in 

instrument  panel,  that  salience  (in  the  form  peripheral  displays;  greater 

of  greater  contrast  with  the  background  and  contrast  with  the 

competing  visual  stimuli)  is  required  for  an  background  is  required. 

onset  to  capture  attention  in  data-rich  Eccentricity  influence 

environments  like  the  modern  flight  deck,  performance 

especially  as  the  indication  in  question  moves 

farther  into  peripheral  vision,  colour-coding  of 

the  2  peripheral  displays  allowed  pilots  to 

make  the  choice  response  without  having  to 

reorient  foveal  visual  attention  to  the  FMA 

(Flight  mode  annunciations) 
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FERNANDEZ 
DUQUE  AND 

THORNTON 

2000 

IMPLICIT  CB/flIcker 

EXP  1  &2:  4x4  GRID  OF  16 
RECTANGLES,  half  horizontal, 
half  vertical,  after  a  blank  ISI 
(interstimulus  Interval)  1  rect 
changes  orientation  90 

degrees.  Response  display: 
2AFC  alternate  force  choice 
showing  2rectangles 

diametrically  opposite  located. 
Participants  were  told  to  take  a 
conservative  approach:  only  If 
they  had  seen  the  rectangle 
change  orientation,  or  a  liberal 
approach:  they  were  told  to 
choose  aware  if  they  have 
noticed  or  felt  any  change 
between  the  displays. 

EXPERIMENT  2:  same  layout, 
use  valid  or  invalid  cues  in  the 
response  display:  either  an 
alphanumeric  character  (&/%) 
or  one  of  two  coloured 
rectangles  (red/green).  EXP3: 

8  or  12  items,  layout  clockface 
design,  in  the  response  display, 
all  Items  remained  visible  but 
the  change  distracter  and  the 
target  changed  hue  to  light 
grey.  EXP  4:  same  layout  as 
exp  3  but  use  of  attentional 
cueing 

Only  29%  of  participants  explicity  report 
changes  using  a  conservative  approach 
compare  to  liberal  approach  45%.  For  the 
conservative  block:  more  likely  to  report 
change  close  to  fixation  (55.5%)  vs.  31% 
peripheral.  More  accurate  in  the  conservative 
block  95%  vs.  85%  liberal  block. 
WHEN  REPORTED  BEING  UNAWARE  OF 
CHANGES:  correct  detection  was  more 

frequent  In  the  conservative  block  than  In  the 
liberal.  Accuracy  was  higher  for  rect.  close 
to  fixation  (60.3%)  vs.  55.1%  peripheral. 
EXP2:  Overall  42%  change  detection,  no 
significance  b/w  type  of  probe 

(colour/character),  type  of  cue  (valld/invalid), 
eccentricity  (near/far)  or  their  interaction. 
EXP3:  set-size  had  a  strong  effect  on 

awareness  responses.  8  items:  52%  reported 
to  be  aware,  12  items:  40%  reported 
awareness.  IN  AWARE  TRIALS  89% 
CHANGE  DETECTION.  Correct  unaware 
selection  (Ml  127  ms)  significantly  faster  than 
Incorrect  ones  (M=1 176msec).  EXP4:  THERE 
WAS  A  SIGNIFICANT  VALIDITY  EFFECT 
FOR  AWARE  TRIALS  BUT  NOT  FOR 
UNAWARE  TRIALS 

Attention  Is  not  ALWAYS 
necessary  for  the 

representation  of  change. 

ARCHAMBAULT, 
O'  CONNELL 

AND  P.  SCHYNS 
1999 

CB 

CATEGORIZATION/ 

flicker 

Participants  had  to  learn  a 
specific  set  of  objects 
(computer  or  mug)  in  a  specific 
level  (Mary's  mug/Peter's 
computer)  or  in  a  general  level. 
EXPERIMENT1,  partic.  Group 

EXPERIMENT  1 :  At  a  specific  level, 
participants  notice  the  change  faster  than  if 
they  learn  it  at  a  general  level. 
EXPERIMENT  2:  rule  out  the  possible 
selective  weighting  Interpretation.  Faster 
detection  if  objects  learned  at  a  specific  level 

CATEGORIZATION 

EFFECT 
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1  learn  computers  at  a  general 
level  and  mugs  at  a  specific 
level.  Group  2  vice  versa. 
EXPERIMENT  2  both  groups 
will  learn  to  categorize  both 
mugs  and  computers  at  both 
the  general  and  the  specific 
level 

2.43  alternations  vs.  4.32  in  a  general  level. 
Therefore,  level  of  categorization  at  which  an 
object  is  learned  can  affect  its  visual 
encoding  and  perception 

Benway  (1999) 
CF  VARAKIN 

2004 

BANNER  BLINDNESS 

web  site  design  In  which  they 
put  a  banner  for  a  training 
course  but  people  did  not  sign 
for  it  because  they  could  not 
find  the  link 

the  banner  even  when  it  was  outstandly 
salient  -  bright  colors,  flashing,  big  fonts  was 
not  seen  by  the  users 

RELATIONSHIP  BETWEEN 
OBJECTS  IN  DISPLAY  -  IF 
RELATED  THEY  SHOULD 
BE  CLOSE  TOGETHER  - 
SPATIALLY  (consistent  with 
PCP).  Users  have 

expectations  of  where  in  the 
display  to  look  =  may  ignore 
most  of  what  is  visible  at 
any  given  time. 

SIMONS, 
FRANCONERI 
AND  REIMER 

2000 

CB  /  gradual  changes 

EXP1 ;  Addition/deletion  of  an 
object  in  an  image.  3 

conditions:  gradual  changes  (a 
movie  presents  a  gradual 
change),  DISRUPTION  flicker 
technique  1st  image  stay  on 
view  for  1 1 .25  sec,  ISI  500  ms, 
and  2nd  image  until  response. 
GUESSING  present  the  image 
and  ask  participants  which 
object  they  think  will  be  deleted. 
EXP  2:  CHANGES  OF 
COLOUR 

EXP1 :  gradual  changes  detected  64.3%, 
disruption  57.4%.  Addition  was  detected 
57.8%,  deletion  detection  61.5%.  The  ease 
with  which  observers  guess  was  not 
correlated  with  the  detection  of  it.  EXP2: 
Detection  of  colour  changes  was  better 
detected  in  the  disruption  41%  than  in  the 
gradual  condition  31%.  Colour  changes  were 
detected  less  often  overall  than 

addition/deletion  changes.  Guessing  where 
the  colour  change  will  happen  was  directly 
correlated  with  detection. 

(OUR  PERCEPTION  IS 
SENSITIVE  TO 

LUMINANCE  CONTRAST 
THAN  TO  ABSOULTUE 
LUMINANCE) 
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